IMPROVED BLOCK REQUEST STREAMING SYSTEM USING SIGNALING OR BLOCK CREATION

专利摘要:
improved block request streaming system using signaling or block creation. a block request system improves the user experience and bandwidth efficiency of such systems, typically using an ingest system that generates data in a form to be served by a conventional file server (http, ftp, or similar), in which the ingest system receives content and prepares it in the form of files or data elements to be served by the file server. the system could include c) sequence control, block request timing and assembly, time-based indexing, variable block sizing, optimal block partitioning, random access point placement control, including through multiple rendering versions, dynamic update of presentation data and/or efficient presentation of live content and time shift.
公开号:BR112012006374B1
申请号:R112012006374-0
申请日:2010-09-22
公开日:2021-07-20
发明作者:Michael G. Luby；Mark Watson；Lorenzo Vicisano；Payam Pakzad；Bin Wang；Ying Chen；Thomas Stockhammer
申请人:Qualcomm Incorporated；
IPC主号:

专利说明:

Cross-References to Correlated Orders
[0001] The present application constitutes a non-provisional patent application claiming priority to the following Interim Applications, each on behalf of Michael G. Luby, et al., all entitled "ENHANCED BLOCK REQUEST STREAMING SYSTEM":
[0002] Provisional U.S. Patent Application Serial No. 61/244,767, filed September 22, 2009;
[0003] Provisional U.S. Patent Application Serial No. 61/257,719, filed November 3, 2009;
[0004] Provisional U.S. Patent Application Serial No. 61/258,088, filed November 4, 2009;
[0005] Provisional U.S. Patent Application Serial No. 61/285,779, filed December 11, 2009; and
[0006] Provisional U.S. Patent Application Serial No. 61/285,779, filed January 20, 2010.
[0007] The present application also claims priority from U.S. Provisional Patent Application Serial No. 61/372,399, filed August 10, 2010, in the name of Ying Chen et al., entitled "HTTP STREAMING EXTENSIONS".
[0008] Each provisional application mentioned above is hereby incorporated by the present reference for all purposes. This description also incorporates by reference, as if they were attached in their entirety to this document, and for all purposes, the following applications/patents of the Applicant:
[0009] U.S. Patent No. 6,307,487 to luby (hereinafter referred to as "Luby I");
[00010] U.S. Patent No. 7,067,829 to Shokrollahi et al. (hereinafter referred to as “Shokrollahi I”)
[00011] U.S. Patent Application Serial No. 11/423,391, filed June 9, 2006, entitled "FORWARD ERROR CORRECTING (FEC) CODING AND STREAMING", in the name of Luby et al. (hereinafter referred to as “Luby II”);
[00012] U.S. Patent Application Serial No. 12/103,605, filed April 15, 2008, entitled "DYNAMIC STREAM INTERLEAVING AND SUB-STREAM BASED DELIVERY", in the name of Luby et al. (hereinafter referred to as “Luby III”);
[00013] U.S. Patent Application Serial No. 12/705 502, filed February 12, 2010, entitled "BLOCK PARTITIONING FOR A DATA STREAM", in the name of Pakzad et al. (hereinafter referred to as ‘Pakzad”); and
[00014] U.S. Patent Application Serial No. 12/859,161, filed August 18, 2010, entitled "METHODS AND APPARATUS EMPLOYING FEC CODES WITH PERMANENT INACTIVATION OF SYMBOLS FOR ENCODING AND DECODING PROCESSES", on behalf of Luby et al. (hereinafter referred to as “Luby IV”) field of invention
[00015] The present invention is related to improved systems and methods for streaming media that are adaptable to network and buffer conditions in order to optimize a streamed media presentation and allow an efficient transport, concomitant or distributed in time, of streaming media data. Fundamentals of the Invention
[00016] Transporting streaming media may become increasingly important as it becomes more common to transport high quality audio and video over packet-based networks such as the Internet, cellular and wireless networks, power grids, and other types of grids. The quality with which the transported media streaming can be rendered can depend on a number of factors, including the resolution (or other attributes) of the original content, the encoding quality of the original content, the capabilities of the receiving devices, and so on. To create a good streaming media experience, the transport and timing of the received signal on receivers can be especially important. A good transport can provide fidelity of the current received at the receiver to what a sender sends, while timing/timing can represent how quickly a receiver can start playing content after an initial request for such content.
[00017] A media transport system can be characterized as a system having media sources or sources, media destinations and channels (in time and/or space) separating the sources and destinations. Typically, a source or source includes a transmitter with access to media in an electronically manageable form, and a receiver with an ability to electronically control the reception of media (or an approximation thereof) and provide it to a media consumer (by example, a user having a display/presentation device coupled in some way to the receiver, a storage element or device, another channel, and so on).
[00018] Although numerous variations are possible, in a common example, a media transport system has one or more servers that have access to media content in electronic form, and one or more client devices or systems make media requests to the servers, and the servers take the media using a transmitter as part of the server, transmitting to a receiver on the client so that the received media can be consumed by the client in some way. As a simple example, there is a server and a client for a given request and response, but this need not be the case.
[00019] Traditionally, media transport/distribution systems can be characterized as a “download” model or a “streaming” model. The download model could be characterized by the independence of timing between transport/delivery of media data and media playback to the user or receiving device.
[00020] As an example, media are “downloaded” or downloaded well in advance of when they will be needed or used, and when they are used, a sufficient quantity will already be available in the receiver. Transport or delivery in the download context is often accomplished by using a file transport protocol, such as HTTP, FTP, or FLUTE (File Delivery Over Unidirectional Transport), and the rate/ distribution speed can be determined by an underlying congestion and/or flow control protocol such as TCP/IP. The operation of the congestion or flow control protocol may be independent of media playback to the target user or device, which may occur concurrently with the download, or at some other time.
[00021] The “streaming” mode can be characterized by a tight coupling between the timing of the distribution of the media data and the reproduction of the media to the user or receiving device. Distribution or transport in such a context is often accomplished through the use of a streaming protocol, such as RTSP (real-time streaming protocol) for control and RTP (real-time transport protocol) for media data. The transport rate can be determined by a streaming server, often in step with the rate/speed of reproduction of the data.
[00022] Some disadvantages of the “download” model may include the fact that, due to timing independence of distribution/transport and playback, media data may not be available when needed for playback (eg due to the fact that the available bandwidth is less than the media data rate), leading to either a momentary interruption of playback ("stall"/"stalling"), which results in an unpleasant experience for the user, or the need downloading the media data well in advance of playback (for example, due to the fact that the available bandwidth is greater than the media data rate), consuming storage resources in the receiving device, which may be few , and consumption of valuable network resources for transport, which will be wasted if the content is not eventually reproduced or used.
[00023] An advantage of the “download” model is that the technology needed to perform such downloads, for example over HTTP, are already well established, are widely used and can be applied in a wide range of applications. Download servers and solutions for the wide scaling of such file downloads (eg HTTP Web Servers and content transport/distribution networks) can be readily available, making it simple and inexpensive to implement services based on such technology. .
[00024] Some disadvantages of the streaming model include the fact that, in general, the media data transport rate is not adapted to the bandwidth available in the server-client connection, and that servers are needed. specialized streaming or a more complex network structure that provides bandwidth and delay guarantees. Although there are streaming systems that support varying the distribution data rate according to available bandwidth (eg AFAS - Adobe Flash Adaptive Streaming), they are generally not as efficient as protocols download transport flow control devices, such as TCP, in utilizing the full bandwidth available.
[00025] Recently, new media transport/distribution systems based on a combination of “streaming” and “download” models have been developed and implemented. As an example, one such model is referred to here as a “block request streaming” model, in which a media client requests blocks of media data from the server infrastructure using a download protocol such as HTTP. A consideration in such systems might be the ability to initiate playback of a stream/stream, for example decoding and displaying audio and video streams using a personal computer and displaying the video on a computer screen/monitor and playing the audio through built-in speakers, or, as another example, decoding and presenting the incoming audio and video streams using a set top box and presenting the video on a television display device and playing the audio through a stereo system.
[00026] Other considerations, such as the ability to decode source blocks quickly enough to keep up with the source's streaming rate, to minimize decoding latency, and to reduce the use of available CPU resources, are concerns. Another consideration lies in providing a stable and scalable streaming distribution solution that allows system components to fail without adversely affecting the quality of streams delivered to receivers. Other problems could occur based on information about a presentation that changes quickly as it is being distributed. Thus, better equipment and equipment are desirable. Brief Summary of the Invention
[00027] A block request streaming system provides improvements in the user experience and bandwidth efficiency of such systems, typically using an ingest system that generates data in a form to be served by a conventional file server (HTTP , FTP, or similar), in which the ingest system absorbs content and prepares it in the form of files or data elements to be served by the file server, which may or may not include a cache. A client device can be adapted to take advantage of the ingestion process as well as include enhancements that provide a better presentation regardless of the ingestion process.
[00028] Such modalities include new enhancements to methods used in a block request streaming client and in a block request ingest system to determine the sequence, timing and assembly of block requests, including the provision of indexing based on time. In some embodiments, new enhancements to methods for assembling blocks and files are provided, including variable block sizing and optimal block partitioning. In some embodiments, new enhancements to random hotspot placement methods are provided, including random hotspot placement across multiple presentation versions. In some embodiments, new enhancements are provided for methods for dynamically updating presentation data, including tagging within metadata. In some embodiments, new enhancements are provided for efficient live content presentation methods and time shift/shift.
[00029] The following detailed description, together with the accompanying drawings, will provide a better understanding of the nature and advantages of the present invention. Brief Description of Drawings
[00030] Figure 1 illustrates elements of a block request streaming system according to embodiments of the present invention.
[00031] Figure 2 illustrates the block request streaming system in Figure 1, showing more details on the elements of a client system that is coupled to a server block infrastructure (BSI) for receiving data that is processed by a content ingestion system.
[00032] Figure 3 illustrates a hardware/software implementation of an ingestion system.
[00033] Figure 4 illustrates a hardware/software implementation of a client system.
[00034] Figure 5 illustrates possible structures for storing the content presented in Figure 1.
[00035] Figure 6 illustrates details of a typical source segment, such as could be stored in the content store illustrated in Figures 1 and 5.
[00036] Figures 7a and 7b illustrate simple and hierarchical indexing within files.
[00037] Figure 8a illustrates variable block sizing with search points aligned in a plurality of versions of a media stream.
[00038] Figure 8b illustrates variable block sizing with non-aligned search points in a plurality of versions of a media stream.
[00039] Figure 9a illustrates a Metadata Table.
[00040] Figure 9b illustrates the transmission of blocks and Table Metadata from the server to the client.
[00041] Figure 10 illustrates blocks that are independent of RAP boundaries.
[00042] Figure 11 illustrates the continuous and discontinuous timing through segments.
[00043] Figure 12 illustrates an aspect of scalable blocks.
[00044] Figure 13 illustrates a graphical representation of the evolution of certain variables in a streaming system by requesting blocks over time.
[00045] Figure 14 illustrates another graphical representation of the evolution of certain variables in a streaming system by requesting blocks over time.
[00046] Figure 15 illustrates a cellular grid of states as a function of threshold values.
[00047] Figure 16 illustrates a flowchart of a process that could be performed in a receiver that can request single blocks and multiple blocks per request.
[00048] Figure 17 illustrates a flowchart of a flexible pipelined process.
[00049] Figure 18 illustrates an example of a set of candidate requests, their priorities and which connections they can be issued, at a given time.
[00050] Figure 19 illustrates an example of a set of request candidates, their priorities and which connections they can be issued, which changed overnight.
[00051] Figure 20 illustrates a flowchart of caching server proxy consistent selection based on a file handle.
[00052] Figure 21 illustrates a syntax definition for a suitable expression language.
[00053] Figure 22 illustrates an example of a suitable hash function.
[00054] Figure 23 illustrates examples of file handle assembly rules.
[00055] Figures 24a to 24e illustrate the bandwidth fluctuations of TCP connections.
[00056] Figure 25 illustrates multiple HTTP requests for repair and source data.
[00057] Figure 26 illustrates an example of channel zapping time with and without FEC.
[00058] Figure 27 illustrates details of a repair segment generator that generates, as part of the ingestion system presented in Figure 1, repair segments from source segments and control parameters.
[00059] Figure 28 illustrates the relationships between source blocks and repair blocks.
[00060] Figure 29 illustrates a procedure for live services at different times on the client.
[00061] In the figures, like items are referenced by like numbers and sub-indices are provided in parentheses to indicate multiple cases of similar or identical items. Unless otherwise indicated, the final sub-index (eg, “N” or “M”) is not intended to limit any specific value and the number of cases for one item may differ from the number of cases for another item , even when the same numbers are illustrated and the sub-index is reused. Detailed Description of the Invention
[00062] As described herein, a goal of a streaming system is to move media from its storage location (or from the location where they are being generated) to a location where they are being consumed, that is, presented to a user or otherwise “used” by a person or an electronic consumer. Ideally, the streaming system can provide uninterrupted playback/playback (or, more generally, uninterrupted “consumption”) at a receiving end and can start playing a stream/stream, or a collection of streams. right after a user has requested the chain or chains. For efficiency reasons, it is also desirable that each current be halted once the user indicates that the current is no longer needed, such as when the user is switching from one current to another current, or when he obeys the presentation of a current , for example, the chain of “subtitles”. If the media component, such as video, continues to be presented, but a different stream is selected to present such a media component, it is often preferred to occupy a limited bandwidth with the new stream/stream and the hold of the previous current.
[00063] A block request streaming system according to the modalities described here provides several benefits. It should be clear that a viable system does not need to include all of the features described here, as some applications could provide a suitably satisfying experience with just a few of the features described here. HTTP streaming
[00064] HTTP streaming is a specific type of streaming. With HTTP streaming, sources can be standard network servers and content delivery networks (CDNs) and can use standard HTTP. Such a technique can involve segmentation and the use of multiple streams/streams, all within the context of standard HTTP requests. Media, such as video, can be encoded at multiple bitrates/bitrates to form different versions or representations. The terms “version” and “representation” are used interchangeably throughout this document. Each version or representation can be broken up or separated into smaller pieces/pieces, each perhaps on the order of a few seconds, to form segments. Each segment can then be stored on a web server/server or CDN as a separate file.
[00065] At the client end, requests can then be made, using HTTP, for individual segments which are amended without gaps by the client. The customer can switch to different data rates based on available bandwidth. The client can also request multiple representations, each presenting a different media component, and can present the media in such representations in a united and synchronous way. Triggers or “triggers” for switching can, for example, include network and buffer occupancy measurements. When operating in regime/steady state, the client can moderate the requests to the server in order to maintain a target buffer occupation.
[00066] Advantages of HTTP streaming can include bit rate adaptation, fast initialization and search, and minimal unnecessary transport/distribution. Such advantages come from controlling the distribution so that it takes only a short time before playback, making maximum use of the available bandwidth (via variable bit rate media) and optimizing stream segmentation and intelligent client procedures.
[00067] A media presentation description can be provided to an HTTP streaming client in such a way that the client can use a collection of files (for example, in formats specified by the 3GPP standard, here referred to as 3GPP segments) to provide a streaming service to the user. A media presentation description, and possibly updates to such a media presentation description, describes a media presentation that constitutes a structured collection of segments, each containing media components such that the client can present the included media in a synchronized manner. and can provide advanced features such as search/search, bitrate switching and joint presentation of media components in different representations. Customer may use the media presentation description information in different ways to provide the service. In particular, from the media presentation description, the client can determine which segments in the collection can be accessed so that the data is useful to the client and user capacity within the streaming service.
[00068] In some embodiments, the media presentation description can be static, although segments can be created dynamically. The media presentation description can be as compact as possible in order to minimize access and download time for the service. Other dedicated server connectivity can be minimized, for example regular or frequent timing synchronization between client and server.
[00069] The media presentation can be mounted to allow access by terminals with different capacities, such as access to different types of access networks, different current network conditions, display sizes, access bitrates and support for CODEC. The client can then extract the appropriate information in order to provide the streaming service to the user.
[00070] The media presentation description can also allow flexibility of implementation and compression as per requirements.
[00071] In a simpler case, each alternate representation can be stored in a single 3GPP file, ie a file as defined in 3GPP TS 26.244, or any other file that conforms to the base media file format ISO as defined in ISO/IEC 14496-12, or derived specifications (such as the 3GPP file format described in 3GPP Technical Specification 26,244). In the remainder of this document, when reference is made to a 3GPP file it should be clear that ISO/IEC 14496-12 and derived specifications can map all features/characteristics to the more general ISO-based file format as defined in ISO /IEC 14496-12 or any derived specifications. The client can then request an initial part of the file to “learn” the media metadata (which is typically stored in the movie header box/film header partition, also referred to as the “moov” box) along with the times and byte offsets of movie fragments. The client can then issue get partial HTTP requests to get movie fragments as required.
[00072] In some arrangements, it may be desirable to split each representation between multiple segments, in that segments. If the segment format is based on the 3GPP file format, the segments contain time “slices” that do not overlap with the movie fragments called “time splicing” or “time-wise splitting”. Each such segment can contain multiple movie fragments, and each can be a valid 3GPP file by itself. In another modality, the representation is divided into an initial segment containing the metadata (typically the movie header/header moov box) and a set of media segments, each containing media data and the concatenation of the initial segment and any media segment forms a valid 3GPP file, as well as the initial segment concatenation and all media segments of a representation form a valid 3GPP file. The entire presentation can be formed by playing each segment in order, mapping the local timestamps/timestamps within the file to the global presentation time according to the initial instant of each representation.
[00073] It should be noted that throughout this description references to a "segment" shall be understood to include any data objects that are fully or partially assembled or read from a storage medium, or otherwise obtained as result of a file download protocol request, including, for example, an HTTP request. As an example, in the case of HTTP, data objects can be stored in real files residing on a disk or other storage medium connected to or part of an HTTP server, or data objects can be mounted via a CGI script, or other dynamically executed program, which is executed in response to the HTTP request. The terms “file” and “segment” are used interchangeably throughout this document unless otherwise specified. In the case of HTTP, the segment can be thought of as the entity body of an HTTP request response.
[00074] The terms “presentation” and “content item” are used interchangeably throughout this document. In many instances, the presentation is an audio, video, or other media presentation that has a defined playtime or “playout” but other variations are possible.
[00075] The terms “block” and “fragment” are used interchangeably in this document unless otherwise specified and generally refer to the smallest aggregate of data that is indexed. Based on available indexing, a client may request different portions or parts of a fragment in different HTTP requests, or may request one or more consecutive fragments or fragment portions in one HTTP request. In the case where segments based on the ISO-based media file format or segments based on the 3GPP file format are used, a fragment typically refers to a movie fragment defined as the combination of a header/header box/box. film fragment (“moof”) and a media data box/box (“mdat”).
[00076] In this document, it is assumed that a network that carries data is based on packets in order to simplify the present descriptions, recognizing that, after reading the present description, those skilled in the art can apply embodiments of the present invention here described to other types of transmission networks, such as continuous bit-stream networks.
[00077] In this document, it is assumed that the FEC codes provide protection against long and variable distribution/transport times of the data, in order to simplify the present descriptions, recognizing that, after reading this description, the technicians in the The area can apply embodiments of the present invention described herein to other types of data transmission problems, such as bit-flip data deterioration. As an example, without FEC, if the last part of a requested fragment arrives much later, or presents a high variance in its arrival time compared to previous parts of the fragment, then the content zapping time can be high and variable, whereas by using FEC and parallel requests, only the majority of data required for a fragment must arrive before it can be retrieved, thereby reducing content zapping time and content zapping time variation. In the present description, it can be assumed that the data to be encoded (ie, the source data) has been separated or broken up into "symbols" of equal length, which can be of any length (up to a single bit), but the symbols could be of different lengths for different pieces of data, for example different symbol sizes could be used for different blocks of data.
[00078] In the present description, and to simplify the descriptions presented here, it is assumed that the FEC is applied to a "block" or "fragment" of data at an instant, that is, a "block" is a "source block" for encoding and decoding purposes. A client device can use the segment indexing described here to aid in determining the source block structure of a segment. Those skilled in the art may apply embodiments of the present invention to other types of source block structures, for example a source block may be a part of a fragment, or encompass one or more fragments or parts of fragments.
[00079] The FEC codes considered for use with request block streaming are typically systematic FEC codes, that is, the source symbols of the source block can be included as part of the source block encoding and thus the source symbols are transmitted. As those in the field will note, the modalities described here apply equally well to FEC codes that are not systematic. A systematic FEC encoder generates, from a source block of source symbols, a number of repair symbols, and the combination of at least some of the repair and source symbols constitutes the encoded symbols that are sent over the channel representing the block. source. Some FEC codes can be useful to efficiently generate as many repair symbols as needed, such as "additive information codes" or "source codes" and examples of such codes include "chain reaction codes" and "chain reaction codes" of multiple stages”. Other FEC codes, such as Reed-Solomon codes, can in practice only generate a limited number of repair symbols for each source block.
[00080] In many such examples it is assumed that a client is coupled to a media server or a plurality of media servers and the client requests streaming media through a channel or a plurality of channels from the media server or the plurality of media servers. However, more involvement arrangements are also possible. Examples of Benefits
[00081] With block request streaming, the media client maintains a coupling between the timing of such block requests and the timing of media playback for the user. Such a model may retain the advantages of the "download" model described above, while avoiding some of the disadvantages that originate in the decoupling of media reproduction from media transport/distribution.. the block request streaming model makes use of the mechanisms of congestion and rate control available in transport protocols such as TCP to ensure that the maximum available bandwidth is utilized for media data. Additionally, dividing the media presentations into blocks allows each block of encoded media data to be selected from a set of multiple encodings available.
[00082] Such selection can be based on various criteria, including matching the media data rate to the available bandwidth, even when the available bandwidth is changing over time, matching the media's resolution or complexity from decoding to the capabilities or configuration of the client, or matching with user preferences, such as language. Selection can also include downloading and showing auxiliary components, such as accessibility components, closed captioning, subtitles, sign language video, and so on. Examples of existing systems that use the block request streaming model include Move Networks™, Microsoft Smooth Streaming, and the Apple iPhone™ Streaming Protocol.
[00083] Commonly, each block of media data can be stored on a server in the form of an individual file, and then a protocol, such as HTTP, is used in conjunction with HTTP server software running on the server, to request the file as a unit. Typically, the client is provided with metadata files, which can be, for example, in XML (Extensible Markup Language) format, or in playlist or playlist text format, or in binary format, which describe media presentation characteristics, such as available encodings (eg, required bandwidth, resolutions, encoding parameters, media type, language), typically referred to as "representations" in this document, and the manner in which encodings were divided into blocks. As an example, the metadata can include a URL (Uniform/Universal Resource Locator) for each block. The URL itself can provide a scheme such as being preceded by the string/string “http://” to indicate that the protocol that should be used to access the documented resource is HTTP. Another example is “ftp://” to indicate that the protocol to use is FTP.
[00084] In other systems, for example, media blocks can be built "on the fly" by the server in response to a request from the client that indicates the part of the media presentation, in time, that is requested. As an example, in the case of HTTP with the “http://” scheme, the execution of such a request response provides a request response that contains some specific data in the entity body of such a request response. The network implementation of how to generate such a request response can be quite different depending on the server implementation serving such requests.
[00085] Typically, each block can be independently decoded. As an example, in the case of video media, each block can start with a “search point” or “seek point”. In some coding schemes, a search point is designated as a “random access point” or RAP (Random Access Point), although not all RAPs can be designated as a search point. Similarly, in other encoding schemes, a search point starts at an Independent Data Refresh (IDR) frame in the case of H.264 video encoding, although not all IDRs can be designated as one search point. A search point is a position in video (or other) media at which a decoder can start decoding without requiring any data regarding previous frames, or data, or samples, such as could be the case where a frame or sample being decoded has been encoded not independently, but eg as the difference between the current/current frame and the previous frame.
[00086] A concern in such systems may be the ability to initiate playback of a stream/stream/stream, for example decoding and displaying incoming audio and video streams using a personal computer and displaying the video on a monitor/screen and play the audio through built-in speakers, or, as another example, decode and present incoming audio and video streams using a “top set box” or decoder and display the video on a television display device and audio playback through a stereo system. A main concern may be to minimize the delay between when a user decides to watch new content received in the form of a stream/stream and takes an action that expresses that decision, for example the user clicks a link inside a window of a browser/browser, or on the “play”/play button/key of a remote control device and, when the content starts playing on the user's screen, hereinafter referred to as the “content zapping instant”. Each of these concerns can be attacked by elements of the extended system described here.
[00087] An example of content zapping occurs when a user is watching a first content distributed via a first stream/stream and then the user decides to watch a second content distributed via a second stream/stream and initiates an action to start watching the second content. The second stream can be sent from the same set as the first stream or from a different set of servers. As another example of content zapping, a user may be visiting a website and decide to start watching content first transported via a first stream by clicking a link within the browser window. Similarly, a user may decide to start playing content not from the beginning, but from a certain point within the stream. The user indicates to his client device that he wants it to look up a position in time and the user could expect the selected instant to be played back instantly. Minimizing content zapping time is important for video playback to allow users a high-quality experience of “surfing” content by browsing and sampling a wide range of available content.
[00088] Recently, it has become common practice to consider the use of early error correction (FEC) codes for the protection of streaming media during transmission. When sent over a packet network, examples of which include the Internet and wireless networks such as those standardized by groups such as 3GPP, 3GPP2 and DVB, the source stream/stream is packaged as it is generated or made available , packets can be used to carry the stream or source content in the order in which it is generated or made available to receivers.
[00089] In a typical application of FEC codes for such types of situations, an encoder can use a FEC code in the creation of repair packages, which are then sent in addition to the original source packages containing the source stream. Repair packets have a property that when a source packet is lost, the repair packets received can be used to recover the data contained in the lost source packets. Repair packages can be used to recover the contents of lost source packages that are entirely lost, but they could also be used to recover partially lost packages, be it fully received repair packages or even partially received repair packages. In this way, fully or partially received repair packages can be used to recover fully or partially lost source packages.
[00090] In more other examples, other types of corruption may occur for the sent data, for example bit values may be changed, and therefore repair packages can be used to correct such corruption and provide as accurate recovery of the packages as possible. source. As other examples, the source stream is not necessarily sent in individual packets, it can instead be sent, for example, in the form of a continuous stream or stream.
[00091] There are several examples of FEC codes that can be used to provide protection to a source stream. Reed-Solomon codes are well known codes for correcting erasures and errors in communication systems. For the correction of erasures in, for example, packet data networks, an efficient, well-known implementation of Reed-Solomon codes uses Cauchy or Vandermonde matrices, as described by L. Rizzo in “Effective Erasure Codes for Reliable Computer Communication Protocols” , Computer Communication Review, 27(2):24-36 (April, 1997) (hereinafter referred to as “Rizzo”) and by Bloemer et al. In "An XOR-Based Erasure-Resilient Coding Scheme", Technical Report TR-95-48, International Computer Science Institute, Berkeley, California (1995) (hereinafter referred to as "XOR-Reed-Solomon") or in other publications.
[00092] Other examples of FEC codes include LDPC codes, chain reaction codes such as those described in Luby I and multistage chain reaction codes such as those in Shokrollahi I.
[00093] Examples of the FEC decoding process for Reed-Solomon code variants are described in Rizzo and XOR-Reed-Solomon. In such instances, decoding can be applied after a sufficient number of repair and source packages have been received. The decoding process can be computationally laborious and, depending on the resources available on the CPU, it can take a considerable amount of time to complete relative to the length of time covered by the media in the block. The receiver can take into account such length of time needed for decoding when calculating the necessary delay between the start of reception of the media stream and the reproduction of the media. Such a delay due to decoding is perceived by the user as a delay between his request for a specific media stream and the start of playback. It is therefore desirable to minimize such a delay.
[00094] In various applications, packets can be further subdivided into symbols on which the FEC process is applied. A packet may contain one or more symbols (or less than one symbol, but usually symbols are not split across packet groups unless the error conditions between packet groups are known to be highly correlated). A symbol can be any size, but often the size of a symbol is at most equal to the size of the packet. Source symbols are those that encode the data to be transmitted. Repair symbols are symbols generated from source symbols, directly or indirectly, and which are added to source symbols (ie, the data to be transmitted can be fully recovered if all source symbols are available and none of the repair symbols are available.
[00095] Some FEC codes may be block-based, in that the encoding operations depend on the symbols that are in a block and may be independent of the symbols that are not in that block. With block-based encoding, a FEC encoder can generate repair symbols for a block from the source symbols in that block, then move on to the next block and need not look up source symbols other than those in the current block. encoded. In a broadcast, a source block containing source symbols may be represented by a coded block comprising coded symbols (which could be some source symbols, some repair symbols, or both). With the presence of repair symbols, not all source symbols are needed in every coded block.
[00096] For some FEC codes, notably Reed-Solomon codes, the encoding and decoding time may become impractical as the number of encoded symbols per source block increases. Thus, in practice, there is often a practical upper limit (255 is an approximate practical limit for some applications) on the total number of encoded symbols that can be generated per source block, especially in a typical case where the encoding process or Reed-Solomon decoding is performed by standard hardware, for example MPE-FEC processes that use Reed-Solomon codes included as part of the DVB-H standard for protection of streams against packet loss are implemented on specialized hardware on a mobile phone that is limited to 255 total Reed-Solomon encoded symbols per source block. Since symbols must often be placed in separate packet payloads/payloads, this imposes a practical upper limit on the maximum length of the source block being encoded. As an example, if the payload/payload of the packet is limited to 1024 bytes or less and each packet carries an encoded symbol, then an encoded source block can have a maximum of 255 kilobytes, which is, of course, an upper limit on the size of the source block itself.
[00097] Other considerations, such as the ability to decode the source blocks quickly enough to keep up with the streaming rate, to minimize the decoding latency introduced by FEC decoding, and to use only a small fraction of the CPU available on the receiving device at any given time. moment in time during FEC decoding, are met by elements described here, as well as dealing with
[00098] The need to provide a stable and scalable streaming transport/distribution solution that allows system components to fail without adversely affecting the quality of streams/streams delivered to receivers.
[00099] A block request streaming system must support changes in the structure or metadata of the presentation, for example changes in the number of available media encodings or changes in media encoding parameters such as bit rate, resolution, aspect ratio, audio or video CODECs or CODEC parameters of changes in other metadata such as URLs associated with content files. Such changes may be necessary for a variety of reasons, including editing/joining content from different sources, such as advertisements or different segments of a larger presentation, modifying URLs or other parameters that become necessary as a result of changes in the server infrastructure by example due to configuration changes, equipment failures, or recovery from equipment failures, or other reasons.
[000100]There are methods in which a performance can be controlled by a continuously updated playlist type file. Since such a file is continuously updated, then at least part of the above changes can be made within such updates. A disadvantage of a conventional method is that client devices must continuously retrieve, what is known as "polling" or "polling", the playlist file exerting a load on the service infrastructure and the fact that such a file does not it can be cached longer than the refresh interval, making the task for the server infrastructure much more difficult. This is solved by the elements described here, so that updates of the type described above are provided without the need for continuous searches by customers for the metadata file.
[000101]Another problem, especially in “live” services, typically known as broadcast distribution, is the user's inability to watch content that was broadcast on broadcast before the user started watching the program. Typically, local personal recordings consume unnecessary local storage, either not possible as the client was not tuned in to the program, or prohibited by content protection rules. Network recording and time-shifted audience are preferred, but require individual user connections to the server and a separate transport protocol and infrastructure from those of “live” services, resulting in duplicated infrastructure and significant server costs. This is also solved by the elements described here. System Overview
[000102] An embodiment of the invention will be described with reference to Figure 1, which shows a simplified diagram of a block request streaming system according to the present invention.
[000103] In Figure 1, a streaming system in blocks 100 is illustrated, comprising the block server infrastructure (BSI) 101, which in turn comprises an ingest system 103 for ingesting content 102, preparing such content and its bundling for service by an HTTP streaming server 104 by storing it in a content store 104. As illustrated, system 100 could also include an HTTP cache 106. In operation, a client 108, such as an HTTP streaming client , sends requests 112 to the HTTP streaming server 104 and receives responses 114 from the HTTP streaming server 104 or HTTP cache 106. In each case, the elements shown in Figure 1 could be implemented, at least in part, in software, it comprises program code that runs on a processor or other electronic components.
[000104]Content may include movies, audio, 2D video/flats, 3D videos, other types of video, images, timed text, timed metadata, or similar. Some content may involve data that must be presented or consumed in a timed manner, such as data for the presentation of ancillary information (station identification, advertisements, stock quotes, Flash™ sequences, and so on).
[000105]As illustrated in Figure 2, media blocks can be stored in a server structure of blocks 101(1), which could be, for example, an HTTP server, a content transport/distribution network device, an HTTP proxy, an FTP server or proxy, or some other system or media server. The server block infrastructure 101(1) is connected to a network 122, which could be, for example, an Internet Protocol (IP) network such as the Internet. A block request streaming client system is presented having six functional components, namely a block selector 123, provided with the metadata described above and performing a function for selecting blocks or partial blocks to be requested among the plurality of available blocks indicated by the metadata, a block requestor 124, which receives request instructions from the block selector 123 and performs the necessary operations to send a request for the specified block, parts of a block, or multiple blocks, to the server infrastructure of blocks 101(1) through network 122 and to receive in response the data comprising the block, as well as a block buffer or accumulator 125, a buffer monitor 126, a media decoder 127 and one or more media transducers 128 that facilitate media consumption.
[000106]The block data received by the block requester 124 is passed to temporary storage to the block buffer 125, which stores the media data. Alternatively, data in received blocks can be stored directly in block buffer 125, as illustrated in Figure 1. Media decoder 127 is provided with media data by block buffer 125 and performs the transformations on the data that are necessary for provide adequate power to 128 media transducers, which present the media in a form suitable for the user or other type of consumption. Examples of media transducers include visual display devices such as those found in cell phones, computer systems, or televisions, and may also include audio display devices such as speakers or headphones.
[000107] An example of a media decoder would be a function that transforms data in the format described in the H.264 video coding standard to analog or digital representations of video frames, such as a pixel map in YUV format with associated timestamps/presentation timestamps for each frame or sample.
[000108] Buffer monitor 126 receives information concerning the contents of block buffer 125 and, based on such information and possibly other information, provides feed to block selector 123, which is used to determine the selection of blocks to order, as described herein.
[000109] In the terminology used here, each block has a "playout time" or "presentation/playback time" or "duration" that represents the amount of time required for the receiver to play the media included in that block at normal speed. In some cases, media playback within a block may depend on having already received data from previous blocks. In rare cases, the playback of some of the media in a block may depend on a subsequent block, in which case the playout time for the block is defined with respect to media that can be played within the block without reference to the subsequent block, and the playing time for the subsequent block is increased by the playing time of the media in such block which can be played only after receiving the subsequent block. Since the inclusion of media in a block that depends on subsequent blocks is a rare case, the remainder of this description will assume that the media in a block does not depend on subsequent blocks, but it should be noted that technicians in the field will know that such variant can be easily added to the modalities described below.
[000110] The receiver may have controls such as “pause”, “fast forward”, “reverse” and so on, which can result in the block being consumed by playback at a rate/ different speed, but if the receiver can obtain and decode each consecutive sequence of blocks in a total time equal to or less than the total playback time excluding the last block in the sequence then the receiver can present the media to the user without stopping or " stalling” (stalling). In some descriptions in this document, a particular position in the media stream is designated as a particular “time” in the media, corresponding to the time that would have elapsed between the start of media playback and the instant when the particular position in the video stream is achieved. Time or position in a media stream is a conventional concept. As an example, when the video stream comprises 24 frames per second, it can be said that the first frame has a position or time of t = 0.0 second and the 241st frame a position of time of t = 10.0 seconds. Of course, in a frame-based video stream, position or time need not be continuous, as each of the bits in the stream from the first bit of the 241st frame to just before the first bit of the 242nd frame could all have the same time value.
[000111] Adopting the above terminology, a block request streaming system (BRSS) comprises one or more clients that make requests to one or more content servers (for example, HTTP servers, FTP servers, and so on) . An ingestion system comprises one or more ingestion processors, where an ingestion processor receives content (in real time or not), processes the content for use by BRSS, and stores it in storage accessible to content servers, possibly at set with metadata generated by the ingest processor.
[000112]BRSS can also contain content caches that coordinate with content servers. Content servers and content caches can be conventional HTTP servers and HTTP caches that receive requests for files or segments in the form of HTTP requests that include a URL, and can also include a range of bytes, so as to request less than the the entire file or segment indicated by the URL. Clients could include a conventional HTTP client that makes requests from HTTP servers and manages the responses to those requests, where the HTTP client is triggered by a new client system that formulates requests, passes them to the HTTP client, gets responses from from the HTTP client and processes these (or stores, transforms, etc.) in order to provide them to a presentation player for playback by a client device. Typically, the client system does not know in advance what media will be needed (as needs may depend on user feeds, changes in user feeds, and so on), so it is described as a “streaming” system in that media are consumed as soon as they are received, or shortly thereafter. As a result, response delays and bandwidth constraints can cause delays in a presentation, such as causing a presentation to pause while the stream reaches the point where the user is consuming the presentation.
[000113]To provide a presentation that is perceived as being of good quality, several details can be implemented in BRSS, either at the client end, at the ingestion end, or both. In some cases the details that are implemented are in consideration of, and to manage the client-server interface on the network. In some modalities, both the client system and the intake system are informed about the innovation or expansion, while in other modalities only one end is informed about the innovation. In such cases, the entire system benefits from the innovation, even if one of the ends is not informed about it, while in others the benefit is only realized if both ends are informed about the innovation, but when one of the ends is not informed it still operates without fail.
[000114]As illustrated in Figure 3, the ingestion system can be implemented in the form of a combination of hardware and software components, according to several modalities. The ingestion system can comprise a set of instructions that can be executed to cause the system to perform one or more of the methods described herein. The system can be assembled in the form of a specific machine like a computer. The system can be a server computer, a personal computer (PC), or any system capable of executing a set of instructions (sequentially or otherwise) that specify the actions to be taken by the system. Furthermore, although only one system is illustrated, the term "system" should also be considered to include any collection of systems that individually or together execute a set (or multiple sets) of instructions to perform any one or more of the methods here. described.
[000115] The ingest system may include the ingest processor 302 (for example, a central processing unit - CPU - central, a memory 304 that can store a program code during execution and storage on disk 306, all of these if communicating with each other via a 300 cabling/bus/bus. the system may also include a 308 video display unit (eg, a liquid crystal display (LCD) or cathode ray tube (CRT)). it may also comprise an alphanumeric feed device 310 (e.g. a keyboard) and a network interface device 312 for receiving source content and broadcasting/distributing content storage.
[000116] Disk storage unit 306 may include a machine-readable medium on which one or more sets of instructions (e.g., software) incorporating any one or more of the methods or functions described herein may be stored. Instructions may also reside, completely or at least partially, within memory 304 and/or within ingest processor 302 during execution thereof by the system, with memory 304 and ingest processor 302 also constituting means for reading by machine.
[000117]As illustrated in Figure 4, the client system can be implemented in the form of a combination of hardware and software components, according to several modalities.
[000118]The client system can comprise a set of instructions that can be executed to make the system perform one or more of the methods described herein. The system can be assembled in the form of a specific machine like a computer. The system can be a server computer, a personal computer (PC), or any system capable of executing a set of instructions (sequentially or otherwise) that specify the actions to be taken by the system. Furthermore, although only one system is illustrated, the term "system" should also be considered to include any collection of systems that individually or together execute a set (or multiple sets) of instructions to perform any one or more of the methods here. described.
[000119] The client system can include the client processor 402 (for example, a central processing unit - CPU - central), a memory 404 that can store a program code during execution and storage on disk 406, all of these communicating each other via a cabling/bus/bus 400. The system can also include a 408 video display unit (eg a liquid crystal display (LCD) or cathode ray tube (CRT)). The system may also comprise an alphanumeric feed device 410 (e.g. a keyboard) and a network interface device 412 for sending requests and receiving responses.
[000120] Disk storage unit 406 may include a machine-readable medium on which one or more sets of instructions (e.g., software) incorporating any one or more of the methods or functions described herein may be stored. The instructions may also reside, completely or at least partially, within memory 404 and/or within client processor 402 during execution thereof by the system, with memory 404 and client processor 402 also constituting machine readable media. Using the 3GPP File Format
[000121]The 3GPP file format or any other file based on the ISO base media file format, such as the MP4 file format, or the 3GPP2 file format, can be used as the containment format for HTTP streaming with the following features. A segment index can be included in each segment to signal time offsets and byte ranges so that the customer can download/download the appropriate pieces of files or media segments as needed. The global presentation timing of the entire media presentation and the local timing within each media segment or 3GPP file can be precisely aligned. The tracks/tracks through the representations can also be aligned by assigning each of them to the global timeline/timeline, in such a way that switching through the representation can occur without interruption and the joint presentation of media components in different representations can be synchronous.
[000122]The file format can contain a profile for Adaptive Streaming with the following properties. All movie data can be contained in movie fragments - the “moov” box/box cannot contain any sample information. The audio and video sample data can be interleaved, with requirements similar to those of the progressive download profile as specified in TS 26.244. The “moov” box can be placed at the beginning of the file, followed by fragment offset data, also referred to as a segment index, containing offset information in byte ranges and time for each fragment, or at least a subset of fragments in the containing segment.
[000123]It may also be possible for the media presentation description to reference files that follow the existing progressive download profile. In such a case the customer can use the media presentation description simply to select the appropriate alternative version from the multiple versions available. Customers can also use HTTP partial get requests with files that conform to the progressive download profile to request subsets of each alternate version and thereby implement a less efficient form of adaptive streaming. In such a case the different representations containing the media in the adaptive download profile can still adhere to a common global timeline so as to allow seamless switching across the representations. Advanced Methods Overview
[000124] In the following sections the methods for the enhanced streaming request blocking systems are described. It should be clear that some of these enhancements can be used with or without other enhancements, depending on the needs of the application. In general operation, a receiver makes requests to a server or other transmitter for specific blocks or portions of data blocks. Files, also called segments, can contain multiple blocks and are associated with a representation of a media presentation.
[000125] Preferably, indexing information is generated, which is also called "segment indexing" or "segment map" which provides a mapping that provides from playout or decoding times to block or fragment byte offsets matches within a segment. Such segment indexing can be included within the segment, typically at the beginning of the segment (at least part of the segment map is at the beginning) and is often small. The segment index can also be provided in a separate segment index or file. Especially in cases where the segment index is contained in the segment, the receiver can download part or all of such segment map quickly and subsequently use it to determine the mapping between time offsets or offsets and the corresponding byte positions of fragments associated with these time offsets within the file.
[000126]A receiver can use byte offset/offset to request data from fragments associated with specific time offsets, without being required to download all data associated with other fragments not associated with the time offsets of interest. In this way, segment mapping or segment indexing can significantly improve a receiver's ability to directly access the portions of the segment that are relevant to the current time offsets of interest, with benefits including better content zapping times, ability to quickly switch from one representation to another as network conditions change, and reducing wasted network resources by downloading media that does not play on a receiver.
[000127] In the case m where switching from one representation (herein referred to as "the representation to which you switch") to another representation (herein referred to as "the representation to which you switch") is considered, the segment index can also be used to identify the initial time or instant of a random access point representation that switches to to identify the amount of data to be requested in the representation from which to switch to ensure that seamless switching is enabled in the sense that the media in the representation from which you change is downloaded within a time of presentation such that the reproduction from which you change is downloaded within a time of presentation such that the reproduction of the representation to which you move can start without interruption from the random access point.
[000128] Such blocks represent segments of the video media or other media/media that the requesting receiver needs to generate the output/emission for the receiver's user. The media receiver can be a client device, such as when the receiver receives content from a server that transmits the content. Examples include set-top boxes or "decoders", computers, game consoles, specially equipped televisions, handheld devices, specially equipped cell phones, or other client receivers.
[000129] Several advanced methods of buffer management are described here. As an example, a buffer management method allows clients to request the highest quality blocks of media that can be received in time to be played back continuously. A variable block size feature improves compression efficiency. The ability to have multiple connections to transmit blocks to the requesting device while limiting the frequency of requests/requests provides improved transmission performance. Partially received blocks of data can be used to continue the media/media presentation. A connection can be reused for multiple blocks without forcing to compromise the starting connection with a given set of blocks. Consistency in server selection across multiple potential servers from multiple clients is improved, which reduces the frequency of duplicate content on nearby servers and increases the likelihood that one server will contain a complete file. Customers can request media blocks based on metadata (such as available media encodings) that are embedded in the URLs for files containing the media blocks. A system can provide for the calculation and minimization of the amount of accumulation/buffer time required before content playback can begin without incurring subsequent media playback pauses. The available bandwidth can be shared among several media blocks, adjusted as the playing time of each block approaches, so that, if necessary, a larger part of the available bandwidth can be allocated to the block with the closest playing time.
[000130]HTTP streaming can employ metadata. Presentation-level metadata includes, for example, stream duration, available encodings (bitrates, codecs, spatial resolutions, frame rates, language/language, media types), pointers/pointers to stream metadata for each encoding, and content protection (digital rights management (DRM) information). Stream metadata can be URLs to segment files.
[000131] Segment metadata can include range/byte range information versus timing information for requests/requests within a segment, and identification of random access points (RAPs) or other search/search points, where part or all of such information may be part of a segment index or segment map.
[000132]Streams/streams can include multiple encodings of the same content. Each encoding can then be divided into segments where each segment corresponds to a storage unit or file. In the case of HTTP, a segment is typically a resource that can be referenced by a URL and requesting such a URL results in the segment reply/return as the entity body of the request response message. Segments can comprise multiple groups of images (GOPs). Each GOP can further comprise multiple fragments where segment indexing provides time/offset or byte offset information for each fragment, ie, the indexing unit is a fragment.
[000133] Fragments or portions of fragments can be requested through parallel TCP connections to increase throughput or transmission/throughput capacity. This can alleviate problems that arise when sharing connections over a choked link or when connections are lost due to congestion, thus increasing overall delivery speed and reliability, which can substantially improve speed and time reliability. content zapping. Bandwidth can be traded off for latency by exaggerated request, but care must be taken to avoid placing requests for segments too far in the future, which can increase the risk of starvation of content.
[000134]Multiple requests per segment to the same server can be chained/pipelined (performing the next request, before the current request is fully serviced) to avoid repetitive TCP startup delays. Requests for consecutive fragments can be aggregated into a single request.
[000135]Some CDNs prefer large files and can trigger minor/background retrievals of an entire file from an origin server upon first observing a range/interval request. However, most CDNs will service requests from the cache if the data is available. Therefore, it can be advantageous to have a portion of customer orders for an entire segment file. Such requests can be later canceled if necessary.
[000136]Valid switching points can be search points, specifically RAPs for example, in the target stream. Different implementations are possible, such as fixed GOP structures or the alignment of RAPs across flows (based on media/media start, or based on GOPs).
[000137]In a modality, segments and GOPs can be aligned through different rate flows. In such a modality, the GOPs can be of variable size and can contain multiple fragments, but the fragments are not aligned between flows of different rates.
[000138]In some modalities, file redundancy can be used with advantages. In such embodiments, an erasure code is applied to each fragment to generate redundant versions of the data. Preferably, the source formatting is not changed due to the use of FEC, and additional repair segments, for example as a representation dependent on the original representation, containing FEC repair data are generated and made available as an additional step in the system. ingestion. The client, which is able to reconstruct a fragment using only source data for that fragment, can only request source data from servers for the fragment within the segment. If the servers are not available, or if the connection to the servers is slow, which can be determined before or after the source data request, additional repair data may be required for the fragment from the repair segment, which decreases the time to reliable delivery of enough data to recover the chunk, possibly through FEC decoding to use a combination of source data and repair data to recover the chunk's source data. In addition, additional repair data may be requested to allow for fragment recovery if a fragment becomes urgent, ie its replay time becomes imminent, which increases the percentage of data for such fragment on a link , but which is more efficient than terminating the other links on the link to free up bandwidth. This can also reduce the risk of missing content due to the use of parallel connections.
[000139]The fragment format can be a stored stream of real-time radio transport protocol (RTP) packets with audio/video synchronization achieved through real-time transport control protocol RTCP.
[000140]The segment format can also be a stored stream of MPEG-2 TS packets with audio/video synchronization achieved through internal MPEG-2 TS timing/timing.
[000141]Using Signaling and/or Blocking to Make Streaming More Efficient
[000142]Various features or characteristics may or may not be used in a block request streaming/streaming system to provide better performance. Performance may be related to the ability to play a presentation without interruption, to obtain media data within bandwidth constraints, and/or to do so within the limited resources of processors on a client, server, and/or ingest system. . Some of these features will now be described. Indexing Within Segments
[000143]To formulate partial GET requests for movie fragments, the client can be informed about the byte offset/offset and start time in decoding or presentation time of all media components contained in the fragments within the file or segment and also which fragments begin with or contain random access points (and are therefore suitable to be used as switching points between alternative representations), where such information is often referred to as segment indexing or segment map. The start time/instant in decoding or presentation instant can be expressed directly or it can be expressed as deltas/differences with respect to a reference time/instant.
[000144]Such time and byte offset indexing information may require at least 8 bytes of data per movie fragment. As an example, for a two-hour movie contained in a single file, with 500 ms movie fragments, this would total about 112 kilobytes of data. Downloading all of this data when starting a presentation can result in a significant additional startup delay. However, the time and byte offset data can be hierarchically encoded so that the client can quickly find a small span of time and offset data relevant to the point in the presentation where he wants to start. Information can also be distributed within a segment such that some refinement of the segment index can be found interspersed with the multimedia data.
[000145]Note that if the representation is time-segmented into multiple segments, the use of the present hierarchical coding may not be necessary, as the complete time and displacement data for each segment may already be quite small. As an example, if the segments are one minute instead of two hours in the example above, the offset byte index/time information constitutes about 1 kilobyte of data, which can typically fit into a single TCP/ packet. IP
[000146]Different options are possible for adding fragment and offset timing data to a 3GPP file:
[000147]First, the Movie Fragment Random Access Box (“MFRA”) can be used for this purpose. MFRA provides a table, which can help readers find random hotspots in a file using movie fragments. In support of this function, the MFRA incidentally contains the byte offsets of MFRA boxes/boxes containing random access points. The MFRA can be placed at or near the edge of the file, but this is not necessarily the case. By searching from the end of the file for a movie fragment random access box and using the size information in it, you can find the beginning of a movie fragment random access box. However, putting the MFRA at the end for HTTP streaming typically requires at least 3 to 4 HTTP requests to access the desired data: at least one to request the MFRA from the end of the file, one to get the MFRA, and, finally, one to get the desired fragment in the file. Therefore, placement at the beginning may be desirable as the mfra can be downloaded together with the first media data in a single request. Also, using MFRA for HTTP streaming can be inefficient as none of the information in the MFRA is needed other than time and moof_offset and specifying offsets/offsets rather than lengths may require more bits.
[000148]Secondly, the item location box/box (iLOC) can be used. "iLOC" provides a directory of metadata resources in this or other files by locating your file containing its offset within that file and its length. As an example, a system could integrate all externally referenced metadata resources into a file, readjusting file offsets/offsets and file references accordingly. However, "iLOC" is intended to provide the location of metadata, so coexistence with real metadata can be difficult.
[000149] Lastly, and perhaps most appropriately, is the specification of a new box/box, known as the Time Index Box (TIDX), specifically dedicated to the purpose of providing exact fragment times/durations and byte offsets in an efficient way. This will be described in more detail in the following section. An alternative box with the same functionality could be the segment index box (SIDX). Here, unless otherwise noted, these two can be interchangeable, given that the two boxes provide the ability to provide the exact times or durations of fragments and byte offsets in an efficient manner. The differences between TIDX and SIDX are given below. It should be clear how to swap the TIDX and SIDX boxes, as both implement a segment index. Segment Indexing
[000150]A segment has an identified start time/time and an identified number of bytes. Multiple fragments can be concatenated into a single segment and clients can issue requests that identify the specific byte range within the segment that corresponds to the desired fragment, or a subset of the fragment. As an example, when HTTP is used as the request protocol, the HTTP range header/header can be used for this purpose. This approach requires the client to have access to a segment “segment index” that specifies the position within the segment of the different fragments. Such a segment index can be provided as part of the metadata. Such an approach has the result that far fewer files need to be created and managed compared to where each block is kept in a separate file. Managing the creation, transfer and storage of a very large number of files (which could extend to several thousand for a presentation of, say, 1 hour) can be complex and error prone, thus reducing the number of files represents an advantage.
[000151]If the client only knows the desired start time of a smaller portion of a segment, it can request the entire file and then read the file in order to determine the appropriate playback start location. To improve bandwidth utilization, segments can include an index file in the form of metadata, where the index file maps the byte ranges of individual blocks to the time ranges the blocks correspond to, which is called such as segment indexing or segment mapping. Such metadata can be formatted as XML data, or it can be binary, following, for example, the atom and box/box structure of the 3GPP file format. Indexing can be simple, where the time and byte ranges of each block are absolute relative to the beginning of the file, or it can be hierarchical, where some blocks are grouped into matrix/“parent” blocks (and those in grandparent blocks and so on) and the time and byte range for a given block is expressed in relation to the time and/or byte range of the block's parent block. Indexing Map Structure Example
[000152] In an embodiment, the original source data for a representation of a media stream may be contained in one or more media files here called "media segment", where each media segment contains the media data used for the playback of a continuous time segment of the media/media, for example 5 minutes from media playback.
[000153]Figure 6 illustrates an example of a structure of a media/media segment. Inside each segment, either at the beginning or spread throughout the source/source segment, there can also be indexing information, which comprises a time/offset map or segment byte offset. The segment time/byte offset map in a modality can be a list of byte time/offset pairs (T (0), B (0)), (T (1), B (1)), . .., (T (i ), B (i)), ..., (T (n), B (n)), where T (i-1) represents a start time within the segment for playback of the 1st media fragment in relation to the initial start time of the media/media between all media segments, T(i) represents an end time for the ith fragment (and therefore the start time for the next fragment), and byte offset/offset B(i-1) is the corresponding byte index of the start of the data within this source segment where the ith media fragment starts relative to the start of the source segment, and B(i) is the byte index corresponding to the end of the 1st fragment (and therefore the index of the first byte of the next fragment). If the segment contains multiple media components, then T(i) and B(i) can be given for each component in the segment in an absolute manner, or they can be expressed in relation to another media component that serves as a component reference. of media/media.
[000154] In this modality, the number of fragments in the source/source segment is n, where n can vary from segment to segment.
[000155]In another modality, the time offset in the segment index for each fragment can be determined with the absolute start time/instant of the first fragment and the durations of each fragment. In this case, the segment index can document the start time of the first fragment and the duration of all the fragments that are included in the segment. The segment index can also only document a subset of the fragments. In this case, the segment index documents the duration of a subsegment that is defined as one or more consecutive fragments, ending either at the end of the containing segment or at the beginning of the next subsegment.
[000156]For each fragment, there can also be a value that indicates whether or not the fragment starts at, or contains a search point, that is, at a point where no media after that point depends on any media prior to this point , and therefore the media/media from the fragment onwards can be played independently from previous fragments. Search/search points are generally the points in the media where playback can start independently of all previous media. Figure 6 also illustrates a simple example of possible segment indexing for a source segment. In this example, the time offset value is in units of milliseconds, so the first fragment of this source segment starts 20 seconds after the media/media start and the first fragment has a play time of 485 milliseconds. The byte offset/offset from the beginning of the first fragment is 0, while the byte offset from the end of the first fragment/beginning of the second fragment is 50245, so the first fragment has a size of 50245 bytes. If the fragment or subsegment does not start with a random access point, but the random access point is contained in the fragment or subsegment, then the decoding time or presentation time difference between the start time and the real RAP instant can be calculated. This allows that in case of switching to this media segment, the client can know the time accurately until the representation switch must be presented.
[000157]In addition to, or in place of simple or hierarchical indexing, chained “daisy wheel” indexing and/or a hybrid indexing could be used.
[000158]Since sample durations for different tracks/tracks can be different (eg video samples can be displayed for 33 ms, while an audio sample can last 80 ms), the different tracks of a fragment Movie clips may not start and end at precisely the same time, meaning the audio may start slightly before or slightly after the video, with the opposite being true for the previous fragment to compensate. To avoid ambiguity, the timestamps/timestamps specified in the byte and time offset data can be specified with respect to a particular track/track and this can be the same track for each representation. Usually this will be the video track. This allows the client to exactly identify the next video frame when he is changing/switching representations.
[000159]Precautions should be taken during presentation to maintain a close relationship between time scales and presentation time, to ensure smooth, smooth playback and maintenance of audio and video synchronization despite the above issue.
[000160]Figure 7 illustrates some examples, such as a simple index 700 and a hierarchical index 702.
[000161] Provided below are two specific examples of a box/box that contains a segment map, one designated as a time index box/box (TIDX) and one referred to as SIDX. The definition follows the box/box structure according to the ISO base media file format. Other schemes for boxes of this type to define a similar syntax and with the same semantics and functionality are known to those skilled in the art. Time Index Box Definition Box Type: “tidx” Container: File Required: No Quantity: Any number zero or one
[000162] The time index box can provide a set of time indices and byte offsets that associate certain regions of the file with certain time intervals in the presentation. The time index box can include a targettype field, which indicates the type of reference data. For example, a time index box with targettype “moof” provides an index to the media fragments contained in the file in terms of both time and bytes offsets/offsets. A time index box with targettype of the time index box can be used to build a hierarchical time index, allowing file users to quickly navigate to the required part of the index.
[000163] The segment index can, for example, contain the following syntax: aligned (8) class TimeIndexBox extends FullBox ('frai') { unsigned int (32) targettype; unsigned int(32) time_reference_track_ID; unsigned int (32) number_of_elements; unsigned int (64) first_element_offset; unsigned int (64) first_element_time; for (i = 1; i <= number_of_elements; i++) { bit (1) random_access_flag; unsigned int (31) length; unsigned int (32) deltaT; } } Semantics: targettype: is the data type of the box referenced by this time index box. This can be either Movie Header/Header Fragment (“Moof”) or Time Index Box (“tidx”). time-reference_track_id: indicates the path/track against which time offsets in this index are specified. number_of_elements: The number of elements indexed by this time index box. first_element_offset: The byte offset from the beginning of the file of the first indexed element. first_element_time: The start time/instant of the first indexed element, using the timescale specified in the track/track media header box identified by time_reference_track_id. random_access_flag: One if the element's start time is a random access point. Otherwise zero. length/length: The length of the indexed element in bytes. deltaT: The difference in terms of the timescale/timescale specified in the track media header box identified by time_reference_track_id between the start time of this element and the start time of the next element. Segment Index Box/Box
[000164] The segment index box (“sidx”) provides a compact index of movie fragments and other segment index boxes in a segment. There are two loop/binding structures in the segment index box. The first loop documents the first sample of the subsegment, that is, the sample in the first movie fragment referenced by the second link. The second link provides an index of the subsegment. The container for the “sidx” box is directly the file or segment. Syntax aligned(8) class SegmentIndexBox extends FullBox('sidx', version 0) { unsigned int (32) reference_track_ID; unsigned int (16) track_count; unsigned int (16) reference_count; for (i = 1; i <= track_count; i++) { unsigned int (32) track_ID; if (version == 0) { unsigned int (32) decoding_time; } Else { unsigned int (64) decoding_time; } } for (i = 1; i <= reference_count; i++) { bit (1) reference_type; unsigned int (31) reference_offset; unsigned int subsegment_duration (32); bit (1) contains_RAP; unsigned int (31) RAP_delta_time; } } Semantics: reference_track_ID gives the track_ID for the reference track or track. track_count: the number of tracks indexed on the next link (1 or higher); reference_count: the number of elements indexed per second link (1 or greater); track_ID: the ID of a track for which a track fragment is included in the first movie fragment identified by this index; exactly one track_ID in this loop is equal to reference_track_ID; decoding_time: the decoding time for the first sample in the track identified by track_ID in the movie fragment referenced by the first item on the second link, expressed in the track's timescale (as documented in the media box header/header timescale field of the trail); reference_type: when set to 0, indicates the reference is to a movie fragment box (moof), when set to 1, indicates the reference is to a segment box/box index (sidx); reference_offset: the distance in bytes from the first byte in the segment index box sequence to the first byte in the referenced box; subsegment_duration: when the reference is to a segment index box, this field carries the sum of the subsegment_duration fields in the second link of that box, and when the reference is to a movie fragment, this field carries the sum of the sample durations of the samples on the reference track, the indicated film fragment, and subsequent film fragments up to the first film fragment documented by the next link entry, or the end of the subsegment, whichever comes first; duration is expressed in the track's timescale (as documented in the timescale field of the track's media header box); contains_RAP: when the reference is to a movie fragment, then this bit can be 1 if the track fragment within that movie fragment for the track with track_ID equal to reference_track_ID contains at least one random access point; otherwise this bit is set to 0; when the reference is to a segment index, then this bit is set to 1 only if any of the references where the segment index has that bit set to 1 and 0 otherwise; RAP_delta_time: if contains_RAP is 1, it provides the presentation time (composition) of a random access point (RAP); reserved with value 0 if contains_RAP is 0. Time is expressed as the difference between the decoding time of the first sample of the subsegment documented by this entry and the presentation time (composition) of the random access point, in the track with equal track_ID to reference_track_ID. Differences between TIDX and SIDX
[000165] SIDX and SIDX provide the same functionality with regard to indexing. The first SIDX link additionally provides the global timing for the first movie fragment, but the global timing can also be contained within the movie fragment itself, either absolutely or relative to the reference track.
[000166] The second SIDX link implements the TIDX functionality. Specifically, SIDX allows you to have a mix of targets for the reference for each index referenced by reference_type, while TIDX only references TIDX or just moof. The number_of_elements in TIDX corresponds to the refere_count in SIDX, the time_reference_track_id in TIDX corresponds to the reference_track_ID in SIDX, the tfirst_element_offset in TIDX corresponds to the reference_offset in the first entry of the second link, first_element_time in TIDX corresponds to decoding_time in TIDX corresponds to the decoding_time of reference_track in the firstaccesslace contains_RAP in SIDX with the additional freedom that in SIDX the RAP may not necessarily be placed at the beginning of the fragment and therefore requires RAP_delta_time, length/length in TIDX matches reference_offset in SIDX, and finally deltaT in TIDX matches the subsegment_duration in SIDX. Therefore, the functionalities of the two boxes are equivalent. Variable Sizing of Blocks and Sub-GOP Blocks
[000167]For video media/media, the relationship between the video encoding structure and the block structure for requests can be important. As an example, if each block starts with a search point, such as a random access point ("RAP"), and each block represents an equal time period of video, then the placement of at least some search point in the media video is fixed and search points will occur at regular intervals within the video encoding. As is well known to those skilled in the art of video coding, compression efficiency can be improved if search points are positioned according to the relationships between video frames, particularly if they are placed in frames that have little in common with previous frames. This requirement that blocks represent equal amounts of time therefore places a restriction on video encoding, such as compression that can be less than ideal or suboptimal.
[000168]It is desirable to allow the position of search points within a video presentation to be chosen by the video encoding system, rather than requiring search points at fixed positions. Allowing the video encoding system to choose search points results in better video compression and therefore higher video media quality can be provided with a given bandwidth, resulting in an improved user experience. Current block streaming request systems may require that all blocks have the same duration (in terms of video time), and that each block start with a search point and this is therefore a disadvantage of existing systems.
[000169] A new block request streaming system will now be described that provides advantages over what was described above. In one embodiment, the video encoding process of a first version of the video component can be configured to select search point positions in order to optimize compression efficiency, but with a requirement that there is a maximum on the duration. between search points. This last requirement restricts the choice of search points by the encoding process and thus reduces the compression efficiency. However, the reduction in compression efficiency is small compared to that imposed if regular fixed positions are needed for the search points, as long as the maximum duration between search points is not too small (eg greater than about about one second). Also, if the maximum duration between seek points is a few seconds, then the reduction in compression efficiency relative to completely free placement of seek points is generally very small.
[000170] In many modes, including the present one, it may occur that some RAPs are not search points, that is, there may be a frame that is a RAP that is between two consecutive search points that is not chosen to be a point for example because the RAP is too close in time to the neighboring search points, or because the amount of multimedia data between the search point before or after the RAP and the RAP is too small.
[000171]The position of search points within all other versions of the media presentation can be restricted to be the same as that of search points in a version (eg the highest data rate) first. This reduces the compression efficiency for these other versions compared to allowing the encoder free choice of search points.
[000172]Using search points typically required a frame to be independently decodable, which generally results in a low compression efficiency for that frame. Frames that do not need to be independently decodable can be encoded with reference to data in other frames, which generally increases the compression efficiency for such frames to a degree that is dependent on the amount of homogeneity between the frame to be encoded and the frames of reference. An efficient choice of search point placement preferably chooses as a search point frame a frame that has little in common with previous frames and thereby minimizes the loss of compression efficiency by encoding the frame in a way that is independently decodable.
[000173]However, the level of homogeneity between a frame and potential frames of reference is highly correlated across different representations of the content, since the original content is the same. As a result, restricting search points in other variants to be the same positions as search points in the first variant does not make a big difference in compression efficiency.
[000174]The search point structure is preferably used to determine the block structure. Preferably, each search point determines the beginning of a block, and there may be one or more blocks that encompass the data between two consecutive search points. Since the duration between seek points is not fixed for encoding with good compression, not all blocks need to have equal playback duration. In some embodiments, blocks are aligned between versions of content - that is, if there is a block spanning a specific group of frames in one version of the content, then there is a block spanning the same group of frames in another version of the content . Blocks of a particular version of content do not overlap and each frame/frame of content is contained exactly within a block of each version.
[000175] A feature that allows the efficient use of variable durations between search points, and therefore GOPs with variable duration, is the indexing of segments or segment map that can be included in a segment or provided by other means to a customer , that is, metadata associated with this segment in this representation that can be provided comprising the start time and duration of each block of the presentation. The client can use this segment indexing data in determining the block in which to start the presentation when the user requests the presentation to start at a particular point that is within a segment. If such metadata is not provided, the presentation can only start at the beginning of the content, or at a random point or close to the desired point (for example, by choosing the starting block by dividing the required starting point (in time) by the duration block average to find the index of the starting block).
[000176]In a modality, each block can be provided as a separate file. In another modality, several consecutive blocks can be aggregated into a single file to form a segment. In this second embodiment, metadata for each version can be provided comprising the start time and duration of each block and the offset/shift of bytes within the file where the block starts. This metadata can be provided in response to an initial protocol request, ie available separately from the segment or file, or it can be contained within the same file or segment as the blocks themselves, for example at the beginning of the file. As will be clear to those skilled in the art, such metadata can be encoded in a compressed form, such as gzip or delta encoding, or in binary form, in order to reduce the network resources needed to transport the metadata to the client.
[000177] Figure 6 shows an example of segment indexing in which the blocks are of variable size and in which the block scope is a partial GOP, that is, a partial amount of multimedia data between one RAP and the next RAP. In this example, the search points are indicated by the RAP indicator, where a RAP indicator value of 1 indicates that the block starts with, or contains, a RAP, or search point, and where a RAP indicator of 0 indicates that the block does not contain RAP or search point. In this example, the first three blocks, ie bytes 0 to 157,033, comprise the first GOP, which has a presentation duration of 1.623 seconds, with a presentation running time from 20 seconds for content at 21.623 seconds. In this example, the first of the first three blocks comprises 0.485 seconds of presentation time and comprises the first 50,245 bytes of multimedia data in the segment. In this example, blocks 4, 5 and 6 comprise the second GOP, blocks 7 and 8 comprise the third GOP, and blocks 9, 10 and 11 comprise the fourth GOP. Note that there may be other RAPs in the multimedia data that are not designated as search points and are therefore not marked as RAPs in the segment map.
[000178]Referring to Figure 6 again, if the customer or receiver wants to access the starting content at the time offset of approximately 22 seconds in the media presentation, then the customer could initially resort to other information, such as the MPD described in larger details later, to first determine that the relevant media data is within this segment. The client can download the first portion of the segment to obtain the segment index, which in this case is only a few bytes, using for example a byte range HTTP request. Using segment indexing, the client can determine that the first block to download is the first block with a time offset that is at most 22 seconds and that starts with a RAP, ie is a search point. In this example, although block 5 has a time offset that is less than 22 seconds, that is, its offset time is 21.965 seconds, segment indexing indicates that block 5 does not start with a RAP and thus instead, based on the indexing segment, the client selects to download block 4 since its start time is at most 22 seconds, that is, its offset time is 21.623 seconds, and it starts with a RAP. So, based on segment indexing, the client will make a gamma range HTTP request starting from byte offset 157,034.
[000179]If segment indexes were not available then the client might have to download all 157,034 bytes of previous data before downloading this data, leading to a much longer initialization time, or longer channel zapping time, and the waste of downloading data that is not useful. Alternatively, if segment indexes are not available, the client can approximate where the desired data starts within the segment, but the approximation can be poor and may miss the appropriate timing and then require rewind, which increases the start delay .
[000180] Generally speaking, each block includes a portion of the multimedia data that, together with the previous blocks, can be played by a media player. Thus, the block structure and signaling of the block segment indexing structure to the customer, either contained within the segment or provided to the customer through other means, can significantly improve the customer's ability to provide fast channel and channel zapping. uninterrupted playback in the face of network variations and interruptions. Supporting variable length blocks and blocks that only encompass parts of a GOP, as enabled by segment indexing, can significantly improve the streaming experience. As an example, referring again to Figure 6 and the example described above, where the client wants to start playback in approximately 22 seconds of presentation, the client can request, through one or more requests, the data within block 4 and, in then feed these to the media player as soon as they are available to start playback. Thus, in this example, playback starts as soon as the 42,011 bytes of block 4 are received at the client, thus allowing for fast channel zapping. If, on the contrary, the client needs to request the entire GOP before playback starts, the channel zapping time would be longer, given that it is 144,211 bytes of data.
[000181] In other modalities, RAPs or search points can also occur in the middle of a block, and there may be data in the indexing segment that indicates where a RAP or search point is located within the block or fragment. In other embodiments, the time offset may be the decoding time of the first frame within the block, rather than the presentation time of the first frame within the block.
[000182] Figures 8(a) and (b) illustrate an example of variable block dimensioning in a structure of search points aligned through a plurality of versions or representations. Figure 8(a) illustrates variable block sizing in a structure of search points aligned across a plurality of versions of a media/media stream, while Figure 8(b) illustrates variable block sizing. block with unaligned search points in a plurality of versions of a media stream.
[000183]The time is shown at the top, in seconds, and the blocks and search points of the two segments for the two representations are shown from left to right in terms of their timing/timing with respect to that timeline and therefore the length of each block shown is proportional to its play time and not proportional to the number of bytes in the block. In this example, segment indexing for both segments of the two representations would have the same time offsets for the search points, but a potentially different amount of blocks or fragments between search points, and the different byte offsets for the blocks due to different amounts of media data in each block. In this example, if the client wants to change from representation 1 to representation 2 in a presentation time of approximately 23 seconds, the client can request up to block 1.2 in representation segment 1 and start requesting the segment for representation 2 starting at block 2.2 and so the switching would occur in the presentation coinciding with search point 1.2 in representation 1, which occurs at the same time as search point 2.3 in representation 2.
[000184]As it should be clear from what has been exposed, the block-on-demand streaming system described does not restrict the video coding to placing search points in specific positions within the content and this mitigates one of the problems of the existing systems.
[000185] In the modalities described above, the system is organized so that the search points of the various representations of the same presentation content are aligned. However, in many cases it is preferable to waive such an alignment requirement. As an example, coding tools are sometimes used to generate representations that do not have the capabilities to generate representations aligned with search points. As another example, content presentation can be coded into different representations independently, without any alignment of search points between different representations. As another example, a representation may contain more search points because it has lower rates and more commonly needs to be switched or it contains search points to support modes with other features such as fast forward or backward or with fast search. Thus, it is desirable to provide methods that make a block request streaming stream capable of dealing efficiently and seamlessly with non-aligned search points across multiple representations for a content presentation.
[000186] In this modality, the positions of the search points through the representations may not be aligned. Blocks are constructed in such a way that a new block starts at each search point and therefore there may not be any alignment between blocks of different versions of the presentation. An example of such a structure of non-aligned search points between different representations is shown in Figure 8(b). The time is shown at the top, in seconds, and the blocks and search points of the two segments for the two representations are shown from left to right in terms of their timing with respect to this timeline and therefore the length of each block shown is proportional to its play time and not proportional to the number of bytes in the block. In this example, segment indexing for both segments of the two representations would have potentially different time offsets for the search points, and also potentially different block or fragment numbers between search points and different byte offsets for the blocks, due to different amounts of media data in each block. In this example, if the client wants to switch from representation 1 to representation 2 in the presentation time of about 25 seconds, the client can request up to block 1.3 in representation segment 1, and start requesting the segment to representation 2 starting in block 2.3 and thus the switching would occur in the presentation coinciding with search point 2.3 in representation 2, which is in the middle of the reproduction of block 1.3 in representation 1, and therefore some of the media/media for block 1.2 do not would be played back (although the multimedia data for the 1.3 block frames that are not played may have to be loaded into the receiver's buffer to decode other 1.3 block frames that are played back).
[000187] In such a mode, the operation of block selector 123 can be modified in such a way that whenever it is necessary to select a block from a representation that is different from the previously selected version, the last block whose first frame does not is chosen. is later than the frame subsequent to the last frame of the last selected block.
[000188] This last described modality can eliminate the need to restrict the search point positions within versions other than the first version and therefore increase the compression efficiency for these versions, resulting in a higher quality presentation for a given range available bandwidth, leading to an improved user experience. Another consideration is that video encoding tools, which perform the function of search point alignment through multiple encodings (versions) of content, may not be widely available and therefore an advantage of the latter modality is that Currently available video encoding tools can be used. Another advantage is that encoding different versions of content can take place in parallel, without any need for coordination between encoding processes for different versions. Another advantage is that additional versions of the content can be encoded and added to the presentation at a later time, without having to provide the encoding tools with lists of specific search point positions.
[000189] Generally speaking, when images are coded as groups of images (GOPs), the first photo/image in the sequence can be a search point, but this need not always be the case. Ideal Block Partitioning
[000190] A concern in a streaming system with block request consists of the interaction between the structure of the encoded media/media, for example, a video media, and the block structure used for block requests. As those skilled in the art of video encoding will know, it is often the case that the number of bits required for the encoded representation of each video frame varies, sometimes substantially, from frame to frame. As a result, the relationship between the amount of data received and the duration of the media encoded by such data may not be simple or straightforward. Furthermore, dividing media data into blocks within a streaming system by requesting blocks adds a new dimension of complexity. In particular, on some systems, a block's media data cannot be played back until the entire block has been received; for example the arrangement of media data within a block or dependencies between media samples within a block from the use of erasure codes may result in such a property. As a result of such complex interactions between block size and block duration and the eventual need to receive an entire block before starting playback, it is common for client systems to adopt a conservative approach where media data is accumulated in buffer before playback starts. Such buffer accumulation results in a long channel zapping time and therefore a poor user experience.
[000191]Pakzad describes "block partitioning methods" which are new and efficient methods for determining how to partition a data stream into contiguous blocks based on the underlying structure of the data stream and further describes several advantages of these methods in the context of a system Streaming or streaming. Another embodiment of the invention for applying Pakzad partitioning methods to a block request streaming system will now be described. This method may comprise arranging the media data so as to be presented in approximate order of presentation time, such that the playing time of any given media data element (eg, a video frame or an audio sample) differs from that of any adjacent media data element by less than a limit provided. The media data so ordered can be considered a data stream in the Pakzad language and any of the Pakzad methods applied to such stream Data streams identify block boundaries with the data stream. Data between any pair of adjacent block boundaries is considered a "block" in the language of such description and the methods of this disclosure are applied to provide presentation of media data within a streaming system by requesting blocks. As will become clear to those skilled in the art from reading this description, the various advantages of the methods disclosed by Pakzad can then be realized in the context of a streaming or streaming system by requesting blocks.
[000192] As described in Pakzad, determining the block structure of a segment, including blocks that encompass partial GOPs or portions of more than one GOP, can impact the customer's ability to allow fast zapping time of channel. In Pakzad, methods are provided that, given a target startup time, would provide a block structure and a target download rate that would ensure that if the client started downloading the representation at any search point and started playback after time If the target initialization has elapsed, playback would continue uninterrupted as long as at each point in time the amount of data the client has downloaded is at least the target download rate, as this provides the client with a means to determine when to start playback of the representation in the earliest possible time, allowing the customer to continue to reproduce the representation as long as the download meets the condition described above. In this way, the method described later provides a means to include the target start time and target download rate in the media presentation description so that it can be used for the purposes described above. Media Presentation Data Model
[000193]Figure 5 illustrates possible structures of the content storage shown in Figure 1, including segments and media presentation description (MPD) files and a breakdown of the segments, timing and other structures within an MPD file. Details of possible implementations of structures or MPD files will now be described. In several examples, the MPD is described as a file, but non-file structures can also be used.
[000194] As illustrated there, the content store 110 maintains a plurality of source segments 510, MPDs 500 and repair segments 512. An MPD may comprise period records 501, which in turn may comprise representation records 502 which contain 503 segment information, such as references to 504 boot segments and 505 media segments.
[000195] Figure 9(a) illustrates an example metadata table 900, while Figure 9(b) illustrates an example of how an HTTP 902 streaming client obtains metadata table 900 and media blocks 904 through a connection to an HTTP streaming server 906.
[000196] In the methods described herein a “media presentation description” is provided which comprises information about the representations of the media presentation that are available to the client. Representations can be alternatives in the sense that the customer selects one of the different alternatives, or they can be complementary in the sense that the customer selects several of the representations, each possibly also from a set of alternatives, and presents them together. Representations can advantageously be assigned to groups, with the client programmed or configured to understand that, for representations in a group, they are each alternative to the others, while representations from different groups are such that more than one representation must be presented together. In other words, if there is more than one representation from a group, the client chooses one representation from that group, one representation from the next group, and so on, to form a presentation.
[000197]The information describing representations may advantageously include details of the applied media CODECs, including profiles and levels of the CODECs that are needed to decode the representation, video frame rates, video resolution and data rates. The client receiving the media presentation description can use that information to determine in advance whether a representation is suitable for decoding or presentation. This represents an advantage, because if the differential information were contained only in the binary data of the representation, it would be necessary to request the binary data of all the representations and analyze and extract the relevant information in order to discover information about its adequacy. These multiple requests and data attachment extraction analysis can take some time which would result in a long boot/boot time and therefore a poor user experience.
[000198]In addition, the media presentation description may include information that restricts customer requests based on time of day. As an example, for a “live” service the customer can be limited to requesting parts of the presentation that are close to the “current broadcast instant”. This represents an advantage since, for live broadcasting, it may be desirable to purge data from the content server infrastructure that has been broadcast more than a given threshold before the current broadcast time. This may be desirable for reusing storage resources within the service infrastructure. This may also be desirable depending on the type of service being offered; for example, in some cases a presentation may only be made available live due to a certain subscription model determined for receiving client devices, while other media presentations may be made available live and "on demand"/on demand, and other presentations they can be made available only live for a first class of client devices, only on demand for a second class of client devices, and a combination of live or on demand for a third class of client devices. The methods described in the media presentation data model (below) allow the customer to be informed of such policies so that the customer can avoid making requests and adjusting offers to the user for data that may not be available in the infra - service structure. As an alternative, for example, the client can present a notification to the user that such data is not available.
[000199] In another embodiment of the invention, the media segments may conform to the media base ISO file format described in ISO/IEC 14496-12 or derived specifications (such as the 3GP file format described in the Specification 3GPP Technique 26,244). The use of the 3GPP File Format section (above) describes new enhancements to the media base ISO file format that enable efficient use of the file format's data structures within a block-on-demand streaming system. As described in this reference, information can be provided within the file which allows for fast and efficient mapping between the time segments of the media presentation and the byte ranges within the file. The media data itself can be structured according to the Film Fragment structure defined in ISO/IEC14496-12. Such information providing offsets/offsets of time and bytes can be structured hierarchically, or as a single block of information. Such information can be provided at the beginning of the file. Providing such information using an efficient encoding as described in the Using the 3GPP File Format section results in the client being able to retrieve this information quickly, using for example a partial HTTP GET type request, in the case where the File download used by the block-on-demand streaming system is HTTP, which results in a short start, search, or stream switching time and therefore an improved user experience.
[000200]The representations in a media presentation are synchronized on a global schedule/timeline to ensure seamless and seamless switching in representations, typically being alternates, and to ensure a synchronized presentation of two or more representations. Therefore, the timing of media samples contained in representations within an adaptive HTTP streaming presentation can be related to a continuous global timeline across multiple segments.
[000201]A block of encoded media containing media of various types, for example audio and video, may have different presentation end times for different types of media. In a block-on-demand streaming system, such blocks of transmission may be played back consecutively so that each type of media is played continuously and therefore media samples of one type of a block may be played before media samples of another type of the previous block, which is referred to here as “continuous block splicing”. Alternatively, such blocks of media may be played in such a way that the first sample of any type of block is played after the last sample of any type of the previous block, which is referred to herein as "discontinuous block splicing". Block seamless splicing may be appropriate when both blocks contain media from the same content item and representation, encoded sequentially, or in other cases. Typically, within a representation, continuous block splicing can be applied when two blocks are spliced. This is advantageous as existing coding can be applied and segmentation can be performed without the need to align media tracks at block boundaries. This is illustrated in Figure 10, where the video stream/stream 1000 includes the block and other blocks, with RAPs such as the RAP 1204. Media Presentation Description
[000202]A media presentation can be thought of as a structured collection of files on an HTTP streaming server. The HTTP streaming client can download enough information to present the streaming service to the user. Alternative representations can be composed of one or more 3GP files or parts of 3GP files conforming to the 3GPP file format or at least conforming to a well-defined set of data structures that can be easily converted from, or to, a 3GP file.
[000203]A media presentation can be described by a media presentation description. The media presentation description (MPD) can contain metadata that the client can use to build appropriate file requests/requests, for example HTTP GET requests, to access the data in a timely manner and to provide the streaming service to the user. The media presentation description can provide enough information for the HTTP streaming client to select the appropriate 3GPP files and file parts. Units that are flagged to the customer as being accessible are designated as segments.
[000204]Among other items, a media presentation description can contain elements and attributes as follows. MediaPresentationDescription element
[000205]An element encapsulating metadata used by the HTTP Streaming client to provide the streaming service to the end user. The MediaPresentationDescription element can contain one or more of the following attributes and elements. Version: version number for the protocol to ensure extensibility. PresentationIdentifier: information such that the presentation can be uniquely identified among other presentations. It can also contain private/hidden fields or names. UpdateFrequency: Media presentation description update frequency, ie how many times the client can reload the actual media presentation description. If not present, the media presentation may be static. Updating the media presentation may mean that the media presentation cannot be cached. MediaPresentationDescriptionURI: URI to date the media presentation description. Stream: Describes the type of stream or media presentation: text, audio, or video. A video stream type can contain audio and can contain text. Service: describes the type of service with additional attributes. Service types can be “live” and “on demand”. This can be used to inform the client that search/search and access beyond the current time is not allowed. MaximumClientPreBufferTime: A maximum amount of time the client can pre-accumulate or reserve streaming media. Such timing can differentiate streaming/streaming from progressive download if the client is restricted to download beyond this maximum pre-buffer time. The value may not be present, indicating that no pre-buffer restrictions apply. SafetyGuardIntervalLiveService: information about the maximum turn-around time of live service on the server. Provides an indication to the customer of information that is already accessible at the present time. Such information may be necessary if the client and server are expected to operate at UTC time and no accurate time synchronization is provided. TimeShiftBufferDepth: information about how far the client can fall back on a live service with respect to the current time. The extension of this visualization in depth, in displaced time, can be admitted without specific changes in the provision of the service. LocalCachingPermitted: This parameter/flag indicates whether the HTTP client can cache downloaded data locally after it has been replayed. LivePresentationInterval: Contains time intervals during which the presentation can be available by specifying StartTimes (start time) and EndTimes (end time). A StartTime indicates the start time of services and EndTime indicates the end time of the service. If end time is not specified, then end time is currently unknown and UpdateFrequency can ensure that clients get access to end time before the actual end time of the service. OnDemandAvailabilityInterval: The presentation interval indicates the availability of the service on the network. Multiple presentation intervals can be provided. The HTTP client may not be able to access the service outside any specified time window. OnDemand range provisioning, additional viewing time shift/offset can be specified. This attribute can also be present for a live service. If present for a live service, the server can ensure that the customer can access the service as an OnDemand service during all availability intervals provided. Therefore, the LivePresentationInterval cannot match or overlap any OnDemandAvailabilityInterval. MPDFileInfoDynamic: Describes the default dynamic assembly/build of media presentation files. More details are provided below. Specifying pattern at the MPD level can avoid unnecessary repetition if the same rules are used for several or all alternative representations. MPDCodecDescription: Describes the main standard media presentation CODECs. More details are provided below. The standard specification at the MPD level can avoid unnecessary repetition if the same CODECs are used for several or all representations. MPDMoveBoxHeaderSizeDoesNotChange: a flag to indicate whether the MoveBox header/header changes size between individual files within the entire media presentation. This flag can be used to optimize downloading and can only be present in case of specific formats, especially those for which segments contain the moov header. FileURIPattern: A pattern used by the client to generate request messages for files within the media presentation. Different attributes allow generation of unique URIs for each of the files within the media presentation. The base URI can be an HTTP URI. Alternative Representation: Describes a list of representations. Alternative Representation Element:
[000206]An XML element that encapsulates all metadata for a representation. The AlternativeRepresentation element can contain the following elements and attributes. RepresentationID: a unique ID for this specific alternate representation within the media presentation. FilesInfoStatic: Provides an explicit list of start/start times and the URI of all files from a single alternate presentation. Static provisioning of the file list may provide the advantage of an accurate time description of the media/media presentation, but it may not be as compact, especially if the alternative representation contains many files. Also, file names can be arbitrary names. FilesInfoDynamic: provides an implicit way to build the list of start times and URI of an alternative presentation. Dynamic file list provisioning can provide the advantage of a more compact representation. If only the starting times sequence is provided, the timing/timing advantages apply here as well, but the filenames must be dynamically constructed based on the FilePatternURI. If only the duration of each segment is given, the representation is compact and may be suitable for use in live/live services, but file generation may be regulated by global time. APMoveBoxHeaderSizeDoesNotChange: A flag that indicates whether the MoveBox header changes size between individual files within the alternative description. This flag can be used to optimize downloading and can only be present in case of specific formats, especially those for which segments contain the moov header. APCodecDescription: describes the main alternative presentation file CODECs. Media Description Element MediaDescription: an element that can encapsulate all metadata for the media that is contained in this representation. Specifically, it may contain information about the tracks/tracks in this alternate presentation, as well as recommended groupings of tracks, if applicable. The MediaDescription attribute contains the following attributes: TrackDescription: An XML attribute that encapsulates all metadata for the media that is contained in this representation. The TrackDescription attribute contains the following attributes: TrackID: a unique ID for the track/track within the alternate representation. It can be used in case the track is part of a grouping description. Bitrate: track/track bitrate. TrackCodecDescription: XML attribute that contains all the attributes in the CODEC used in this track. The TrackCodecDescription attribute contains the following attributes: MediaName: an attribute defining the media type. Media types include “audio”, “video”, “text”, “application” and “message.” Codec: CodecType including profile and level LanguageTag: LanguageTag/language tag/language if applicable MaxWidth/MaxHeight : for video, the height and width of video content in pixels SamplingRate: for audio sampling rate GroupDescription: an attribute that provides a recommendation to the customer for appropriate grouping based on different parameters GroupType: a type based on in which the customer can decide how to group tracks.
[000207]The information in a media presentation description is advantageously used by an HTTP streaming client to execute requests for files/segments or parts thereof at appropriate times, selecting the segments from suitable representations that match their resources, by example in terms of access bandwidth, display capabilities, codec features and so on, as well as user preferences such as language and so on. In addition, as the presentation media description describes representations that are time-aligned and mapped to a global timeline/timeline, the client can also use the information in the MPD during an ongoing multimedia presentation to initiate appropriate actions to switch between representations, present representations together or search within the media presentation. Signaling the Start Times/Instants of Segments
[000208]A representation can be divided, in terms of time, into several segments. There is an inter-track timing issue between the last fragment of one segment and the next fragment of the next segment. Furthermore, there is another problem of synchronization in the case of using segments of constant duration.
[000209]Using the same duration for each segment can have the advantage that the MPD is compact and static. However, each segment can still start at a random hotspot. Thus, video encoding may be restricted to provide random hotspots at these specific points, or actual segment durations may not be exactly as specified in the MPD. It may be desirable that the streaming system does not place unnecessary restrictions on the video encoding process and so the second option may be preferred.
[000210]Specifically, if the file duration is specified in the MPD as d seconds, then the file # can start with the random access point at or immediately after the time/instant (n-1)°d.
[000211] In this approach, each file can include information such as the exact start time of the segment in terms of overall presentation time. Three possible ways to signal this include: (1) First, restrict the start time of each segment to the exact time/instant as specified in the MPD. But then the media encoder might not have any flexibility about the placement of the IDR frames and might require special encoding for the streaming file. (2) Second, add the exact start time for the MPD for each segment. In the case of demand, the compactness of the MPD can be reduced. For the live case, this may require a regular MPD update, which can reduce expandability. (3) Third, add the global time or exact start time relative to the advertised start time of the representation or the advertised start time of the segment in the MPD for the segment in the sense that the segment contains this information. This can be added to a new box/box dedicated to adaptive streaming. This box can also include the information provided by the "TIDX" or "SIDX" box. A consequence of this third approach is that, when looking for a specific position at the beginning of one of the segments, the client can, based on the MPD, choose the segment after the one that contains the necessary search point. A simple answer in this case might be to move the search point forward to the beginning of the retrieved segment (ie, to the next random access point after the search point). Typically, random access points are provided at least every few seconds (and there is often little coding gain in making them less frequent) and so in the worst case the search point can be moved to occur a few more seconds. later than specified. Alternatively, the client could determine to retrieve the header information for the segment for which the requested search point is actually in the previous segment and request that segment instead. This can result in an occasional increase in the time required to perform the seek operation. List of Accessible Segments
[000212]The media presentation comprises a set of representations, each of which provides some different version of the encoding of the original media content. The representations advantageously contain information about the differentiating parameters of the representation compared to the other parameters. They also contain, explicitly or implicitly, a list of accessible segments.
[000213]Segments can be differentiated between timeless or untimed segments containing only metadata and media segments containing mostly media data. The media presentation description (MPD) advantageously identifies and assigns different attributes to each of the segments, either implicitly or explicitly. Attributes advantageously assigned to each segment comprise the period during which a segment is accessible, the resources and protocols through which the segments are accessible. Additionally, media segments are advantageously given attributes such as the segment start time of the social media presentation and the segment duration of the media presentation.
[000214]When the media presentation is of the "on demand" type, as advantageously indicated by an attribute in the media presentation description, such as OnDemandAvailabilityInterval, the media presentation description usually describes the entire segments and also provides indication when the segments are accessible and when the segments are not accessible. Segment start times are advantageously expressed with respect to the start of the multimedia presentation such that two clients starting playback of the same media presentations, but at different times, can use the same media/media presentation description, as well as the same media segments. Advantageously, this improves the ability to cache segments.
[000215] When the media presentation is of the "live" type, as advantageously indicated by an attribute in the media presentation description, such as the Service attribute, the segments that comprise the media presentation beyond the actual time of day do not they are generally generated or at least not accessible despite the segments being fully described in the MPD. However, with the indication that the media presentation service is of the live type, the customer can produce a list of accessible segments along with the sync attributes for the NOW customer's internal time in terms of hour/clock time conventional, based on the information contained in the MPD and the MPD download time. The server advantageously operates in the sense that it makes the resource accessible, in such a way that a reference client, operating with the MPD instance at the NOW clock time, can access the resources.
[000216]Specifically, the reference client produces a list of accessible segments along with the sync attributes for a NOW client's internal time in clock time, based on the information contained in the MPD and the download time of the MPD. As time progresses, the client will use the same MPD and will create a new list of accessible segments that can be used to continuously play the media presentation. Therefore, the server can advertise segments in an MPD before those segments are actually accessible. This is advantageous as it reduces the frequency of MPD updates and downloads.
[000217]Let's assume that a list of segments, each with start time, tS, is described explicitly by a playlist in elements such as FileInfoStatic or implicitly, using an element such as FileInfoDynamic. An advantageous method for generating a segment list using FileInfoDynamic will be described later. Based on this construction rule, the client has access to a list of URIs for each representation, r, here referred to as FileURI(r,i), and a start time, tS(r,i), for each segment with index i.
[000218]Using information in the MPD to create the segment accessible time window can be performed using the following rules.
[000219]For an "on demand" type service, advantageously indicated by an attribute such as Service, if the clock time on the client is now within any availability range, advantageously expressed by an MPD element such as OnDemandAvailabilityInterval, all segments described in this On Demand presentation are accessible. If the current clock time on the client is outside any range of availability, then none of the segments described in this On Demand presentation will be accessible.
[000220]For a "live" type service, as advantageously indicated by an attribute such as Service, start time, tS(r,i), advantageously expresses the availability time in clock hour. Availability start time can be derived as a combination of live event time/service instant and some server turn around time for capture, encoding and publishing. The time for this process can, for example, be specified in the MPD, for example using a tG security guard interval specified by as SafetyGuardIntervalLiveService in the MPD. This would provide the minimal difference between UTC time and data availability on the HTTP streaming server. In another modality, the MPD explicitly specifies the segment availability time in the MPD without providing the turnaround time in the form of a difference between the live event time and the turnaround time. The descriptions that follow assume that any global times are specified as availability times. Technicians in the field of live media broadcast can derive such information from appropriate information in the media presentation description after reading this description.
[000221]If the current clock time in the client, NOW, is outside any range of the live presentation interval, advantageously expressed by an MPD element such as LivePresentationInterval, none of the segments described in this live presentation are accessible. If the current NOW clock time on the client is within the live presentation range, at least some segments of the segments described in this live presentation may be accessible.
[000222]The restriction of accessible segments is governed by the following values:
[000223]The NOW clock time (as available to the customer).
[000224]The allowed depth/reach for the tTSB time shift buffer, for example specified as TimeShiftBufferDepth in the media presentation description.
[000225] A customer at the relative event time tl can only request segments with start times tS(r, i) in the range of (NOW - tTSB) and NOW (now) or in an interval such as the end time of the segment with duration d is also included, resulting in a range of (NOW - tTSB d) and NOW. Updating MPD
[000226] In some modalities, the server does not know in advance the file or segment locator and the start times/instants of the segments, given that, for example, if the server's location will change, or if the media presentation includes some advertisement/advertisement from a different server, or the duration of the media presentation is unknown, or the server wants to hide the locator for the segments to follow.
[000227] In such modalities, the server could only describe segments that are already accessible or that will be accessible soon after the publication of this instance of the MPD. Furthermore, in some modalities, the client advantageously consumes media close to the media/media described in the MPD, in such a way that the user experiences the contained media program as closely as possible for the generation of media content. As soon as the client anticipates reaching the end of the media segments described in the MPD, it advantageously requests a new instance of the MPD to continue playing in the expectation that the server has published a new MPD describing new media segments. The server advantageously generates new instances of the MPD and updates the MPD on which customers can rely on procedures for continuous updates. The server can adapt its MPD update procedures along with generating segments and publishing the procedures of a reference client that acts as an ordinary client would.
[000228]If a new instance of MPD only describes a short advance in time then customers should frequently request new instances of MPD. This can result in scaling or expansion issues and unnecessary uplink and downlink traffic due to unnecessarily frequent requests.
[000229] Therefore, it is pertinent on the one hand to describe segments as much as possible in the future without necessarily making them readily accessible, and on the other hand to enable unplanned MPD updates to express new server locations, allow insertion of new content, such as advertisements, or provide changes to codec parameters.
[000230]Also, in some modalities, the duration of the media segments may be small, such as in the range of a few seconds. Duration of media segments is advantageously flexible in order to adjust to suitable segment size which can be optimized for transport or cache properties, to compensate for end-to-end delay in live/live services, or other aspects that deal with with storage or delivery of segments, or for other reasons. Especially in cases where segments are small compared to the media presentation period, then a significant amount of media segment features and start times/instants will need to be described in the media presentation description. As a result, the size of the media presentation description can be large, which can negatively affect the download time in the media presentation description and therefore affect the start/start delay of the multimedia presentation and also the use of bandwidth on the access link. Therefore, it is advantageous not only to allow description of a list of media segments using playlists, but also to allow description using templates or URL construction rules. In this description, the terms templates and URL construction rules are used interchangeably.
[000231]Furthermore, models can advantageously be used to describe segment locators in live cases beyond the current time. In such cases, MPD updates are in themselves unnecessary, as the locators as well as the maneuver list are described by the models. However, unforeseen events can still occur, requiring changes in the description of representations or segments. Changes to an adaptive HTTP streaming/streaming media presentation description may be necessary when content from several different sources is amended, for example when an advertisement is inserted. Content from different sources can differ in a variety of ways. Another reason, during live presentations, is that it may be necessary to change the URLs used for content files to predict fail-over/repass in case of failure, from a live source from one server to another.
[000232]In some modalities, it is advantageous that, if the MPD is updated, updates to the MPD are carried out so that the updated MPD is compatible with the previous MPD in the sense that the reference client and therefore any The implemented client generates an identically functional list of accessible segments from the updated MPD for any time up to the expiration time of the previous MPD, just as it would have done from the previous instance of MPD. This requirement ensures that (a) customers can immediately start using the new MPD without syncing with the old MPD, as it is backward compatible with the old MPD before upgrade time; and (b) the update time does not need to be synchronized with the time the actual switch to MPD occurs. In other words, updates to the MPD can be announced in advance, and the server can replace the old instance of the MPD after new information becomes available, without having to maintain different versions of the MPD.
[000233]There can be two possibilities for media synchronization through an MPD update for a set of representations or all representations. Either (a) the existing global schedule continues throughout the MPD update (herein referred to as a "continuous MPD update"), or (b) the current timeline/schedule ends and a new schedule begins with the segment following the change (herein referred to as a “discontinuous MPD update”).
[000234]The difference between these options can be evident when considering that the tracks/tracks of a media fragment, and therefore a segment, usually do not start and end at the same time because of the different grain/sample resolution between the tracks. During normal performance, samples from one track of a fragment may be processed before some samples from another track from the previous fragment, ie there is some kind of overlap between fragments even though there is no overlap within a single track.
[000235]The difference between (a) and (b) is whether such an overlap can be allowed through an MPD update. When the MPD update occurs because of splicing/splicing of completely separate content, such an overlap is generally difficult to achieve, as the new content needs recoding to be spliced with the previous content. Therefore, it seems advantageous to enable the ability to discontinuously update the media presentation by restarting the schedule for certain segments and possibly also defining a new set of representations after the update. Furthermore, if the content has been independently encoded and segmented, then it is also avoided to adjust timestamps/timestamps to fit the global schedule of the previous piece of content.
[000236]When updating occurs for minor reasons, such as just adding new media segments to the list of media segments, or if the location of URLs is changed, then rollovers and continuous updates may be allowed.
[000237]In the case of a discontinuous MPD update, the timeline of the last segment of the previous representation ends at the end of time of the most recent performance of any sample in the segment. The timeline of the next representation (or, more precisely, the first instant of presentation of the first media segment of the new part of the media/media presentation, also referred to as the new period) typically and advantageously starts at the same instant as the end of the presentation of the last period such as to ensure a perfect and continuous reproduction.
[000238]The two cases are illustrated in Figure 11.
[000239]It is preferred and advantageous to restrict MPD updates to segment boundaries. The logic for limiting such changes or updates to segment boundaries is as follows. First, changes to the binary metadata for each representation, typically the movie header, can take place at least on segment boundaries. Second, the media presentation description can contain the pointers/pointers (URLs) to the segments. In a sense, the MPD is the “umbrella”/comprehensive data structure, which gathers all the segment files associated with the media presentation. To maintain this confinement relationship, each segment can be referenced by a single MPD and when the MPD is updated it is advantageously only updated on a segment boundary.
[000240] Segment boundaries do not generally need to be aligned, but for the case of spliced content from different sources and discontinuous MPD updates, it generally makes sense to align the segment boundaries (specifically, that the last segment of each representation can end in the same video frame and cannot contain audio samples with a presentation start time later than the frame presentation time). A discontinuous update can then start a new set of representations each time at a common time, designated as period. The validity start time of this new set of representations is given, for example, by a period start time. The relative start time of each representation is reset to zero and the period start time places the set of representations in this new period on the global media presentation timeline.
[000241]For continuous MPD updates, segment boundaries are not required to be aligned. Each segment of each alternative representation can be governed by a single media presentation description and thus the update prompts for a new instance of a media presentation description, usually triggered by the anticipation that no additional media segments are described in the MPD in operation can be performed at different times depending on the set of representations consumed, including the set of representations expected to be consumed.
[000242]To support updates to MPD elements and attributes in a more general case, any elements, not just representations or set of representations, can be associated with a validity time. Thus, if certain elements of the MDP need to be updated, for example when the number of observations is changed, or URL construction rules are changed, such elements can be updated individually, at specified times, providing multiple copies of the element with times of disjoint validity.
[000243]The validity is advantageously associated with the global media time, such that the described element associated with a validity time is valid in a period of the global schedule/timeline of the media presentation.
[000244] As described above, in one modality the validity times are only added to a complete set of representations. Each complete set then forms a period. The validity time then forms the start time of the period. In other words, in a specific case of using the validity element, a complete set of representations can be valid for a period of time, indicated by an overall validity time of a set of representations. The validity time of a set of representations is referred to as a period. At the beginning of a new period, the validity of the previous set representation expires and the new set of representations is valid. Note again that period validity times are preferably separate or disjointed.
[000245]As mentioned above, media presentation description changes occur at segment boundaries and so, for each representation, the change of one element actually occurs at the next segment boundary. The client can then form a valid MPD including a list of segments for each instant of time within the media submission deadline.
[000246] The splicing of discontinuous blocks may be suitable in cases where blocks contain media data of different representations, or of different content, for example of a content segment and an advertisement, or in other cases. It may be necessary in a block-requested streaming system that changes to the presentation metadata only occur at the block boundaries of the streaming system. This can be advantageous for implementation reasons because updating media decoder parameters within a block can be more complex than updating them only between blocks. In this case, advantageously, it can be specified that validity intervals as described above can be interpreted as approximate, such that an element is considered valid from the first block boundary not earlier than the start of the validity interval specified for the first block boundary, not before the expiry of the specified validity interval.
[000247] An exemplary modality of what was described above describes new improvements to a streaming system by requesting blocks described later in the section Changes in Media Presentations. Segment Duration Signaling
[000248]Discontinuous updates effectively divide the presentation into a series of disjointed intervals, referred to as periods. Each period has its own timeline for timing media samples. Representation media timing within a period can be advantageously indicated by specifying a separate compact list of segment durations for each period or for each representation within a period.
[000249]An attribute, for example referred to as start or start time period, associated with elements within the MPD, can specify the validity time of certain elements within the media submission deadline. This attribute can be added to any elements (attributes that can be assigned a validity can be changed to elements) of the MPD.
[000250]For discontinuous MPD updates the segments of all representations can end in the discontinuity. This generally implies at least that the last segment before the discontinuity has a different duration than the previous ones. The duration of the signaling segment may involve indicating whether all segments are of the same duration or indicating a separate duration for each segment. It is desirable to have a compact representation for a list of segment durations that is efficient in cases where many of them are of the same duration.
[000251]The durations of each segment in a representation or a set of representations can advantageously be done with a single string/string that specifies all segment durations for a single interval since the start of the discontinuous update, ie, the beginning of the period until the last media segment described in the MPD. In one embodiment, the format of this element is a production-conforming text string/string that contains a list of segment duration entries, where each entry contains a duration attribute dur and an optional mult attribute multiplier, indicating that this representation contains <mult> of the first entry segments of duration <dur> of the first entry, then <mult> of the second entry segments of duration <dur> of the second entry, and so on.
[000252]Each duration entry specifies the duration of one or more segments. If the value of <dur> is followed by the “*” character and a number, that number specifies the number of consecutive segments of that duration, in seconds. If the multiplier sign “*” is absent, the number of segments is one. If the “*” is present without any number then all subsequent segments have the specified duration and there cannot be any other entries in the list. As an example, the character string “30*” means that all segments have a duration of 30 seconds. The string “30*12 10.5” indicates 12 segments of duration of 30 seconds, followed by one of duration of 10.5 seconds.
[000253]If segment durations are specified separately for each alternate representation, then the sum of segment durations within each range can be the same for each representation. In the case of video tracks/tracks, the range can end on the same frame in each alternate representation.
[000254] Persons skilled in the art, when reading the present description, can find similar and equivalent ways to express the segment durations in a compact way.
[000255]In another modality, the duration of a segment is signaled as being constant for all segments of the representation except for the last one by a <duration> signal duration attribute. The duration of the last segment before a discontinuous update can be shorter as long as the start point of the next discontinuous update or the start of a new period is given, which then implies the duration of the last segment reaching the start of the next period . Representation Metadata Changes and Updates
[000256] Indications of encoded binary representation metadata changes such as changes to the movie header "moov" can be done in different ways: (a) there can be a moov box/box for all representations in a separate file listed in MPD, (b) there can be a moov box for each alternative representation in a separate file listed in each alternative representation, (c) each segment can contain a moov box and is therefore self-contained, (d) there can be a box moov for all representations in a 3GP file along with MPD.
[000257] Note that in cases (a) and (b), the single "moov" may be advantageously combined with the validity concept above in a sense that more "moov" boxes can be referenced in an MPD, since that their validity is disconnected. As an example, with the definition of a period limit, the validity of the “moov” in the old period can expire with the beginning of the new period.
[000258]In the case of option (a), the reference to the single moov box can be assigned a validity element. Multiple presentation headers can be allowed, but only one can be valid at a time. In another embodiment, the validity time of the entire set of representations in a period, or the entire period, as defined above, can be used as a validity time for such representation metadata, usually provided in the form of the header moov.
[000259]In the case of option (b), if the reference to the moov box of each representation can receive a validity element. Multiple impersonation headers can be allowed, but only one can be valid at a time. In another embodiment, the validity time of the entire representation or the entire period as defined above can be used as a validity time for this representation metadata, usually provided as the moov header.
[000260]In the case of option (c), no signaling can be added in the MPD, but additional signaling in the media stream can be added to indicate whether the moov box will switch to any of the nearby segments. This will be explained further in the context of "Tagging Updates Within Segment Metadata". Flagging Updates Within Segment Metadata
[000261]To avoid frequent updates to the media presentation description, to gain knowledge about possible updates, it is advantageous to flag any such updates along with the media segments. An additional element or elements may be provided within the media segments themselves, which may indicate that updated metadata, such as the media presentation description, is available and must be accessed within a certain period of time to successfully continue the creation of accessible segment lists. In addition, such elements can provide a file identifier, such as a URL, or information that can be used to construct a file identifier, for the updated metadata file. The updated metadata file may include metadata the same as that provided in the original metadata file for the submission modified to indicate ranges of validity along with additional metadata also accompanied by ranges of validity. Such an indication may be provided in media segments of all representations available for a media presentation. A client that accesses a streaming system by block requests, upon detecting this indication within a block of media, can use the file download protocol or other means to retrieve the updated metadata file. The customer is thus provided with information about changes in the media presentation description and the time when they will or did occur. Advantageously, each customer requests the updated media presentation description only once when such changes occur rather than “probing” and receiving the file many times for possible updates or changes.
[000262]Examples of changes to adding or removing representations, changes to one or more representations, such as change in bit rate, resolution, aspect ratio, included tracks/tracks or codec parameters, and changes to the construction rules of URL, for example a different origin server for an ad. Some changes might only affect the startup segment, such as the movie header/header (moov) “atom” associated with a representation, while other changes might affect the media presentation description (MPD).
[000263] In the case of on-demand content, these changes and their timing may be known in advance and could be signaled in the media presentation description.
[000264]For live/live content, changes cannot be known up to the point where they occur. One solution is to allow the media presentation description available at a specific URL to be dynamically updated and require customers to regularly request such an MPD to detect changes. This solution has a disadvantage in terms of scalability (load efficiency and the cache of the origin server). In a scenario with a large number of viewers, caches can receive many requests for the MPD after the previous version has expired from the cache and before the new version is received and everyone can be forwarded to the origin server. The origin server may need to constantly process cache requests for each updated version of the MPD. Also, updates cannot be easily time-aligned with media presentation changes.
[000265]Given that one of the advantages of HTTP streaming/streaming is the ability to use standard web infrastructure and services for scaling, a preferred solution may involve only “static” files (ie cache) and not rely on of clients to “poll” files to see if they have changed.
[000266] Solutions are discussed and proposed to solve metadata updating, including media presentation description and representation binary metadata such as moov atoms in an adaptive HTTP streaming media presentation.
[000267]For the case of live content, the points at which the MPD or moov can be changed might not be known when the MPD is built. Since frequent polling of the MPD for updates should generally be avoided, for reasons of bandwidth and scaling, updates to the MPD can be indicated “in-band” in the segment files, ie each media segment may have the option to indicate updates. Depending on the segment formats (a) to (c) above, different updates may be flagged.
[000268] Generally speaking, the following indication can advantageously be provided in a signal within the segment: an indicator that the MPD can be updated before requesting the next segment within this representation or any following segment that has a higher start time than the start time of the current segment. The update can be announced in advance, indicating that the update only needs to happen, at the latest, in any next segment. This MPD update can also be used to update binary representation metadata such as movie headers/headers in case the media segment locator is changed. Another sign may indicate that with the completion of this segment, no further segments should be requested to move forward in time.
[000269]If the segments are formatted according to the segment format (c), ie each media segment can contain auto-initialized metadata metadata, such as the movie header, then another signal can be added indicating that the subsequent segment contains an updated movie header (moov). This advantageously allows the movie header to be included in the segment, but the movie header only needs to be requested by the customer if the previous segment indicates an update of the movie header or in the case of search or random access when switching representations. In other cases, the client can issue a byte range request for the segment that excludes the movie header from the download, thus advantageously saving bandwidth.
[000270]In yet another modality, if the MPD update indication is flagged, then the sign can also contain a locator as the URL for the updated description of the media presentation. The updated MPD can describe the presentation both before and after the update, using the validity attributes as a new and old period in case of discontinuous updates. This can advantageously be used to allow time-shifted display as described later on, but it also advantageously allows the MPD update to be signaled at any time before changes take effect on it. The customer can immediately download the new MPD and apply it to the ongoing presentation.
[000271] In a specific realization, the signaling of any change to the media presentation description, moov headers, or at the end of the presentation, may be contained in an information box/box formatted according to segment format rules using the ISO base media file format box structure. This box can provide a specific signal for any of the different updates. Streaming Info Box/Box Definition Box type: “sinf” Container: none Required: no Quantity: Zero or one.
[000272]The streaming information box contains information about the streaming/streaming presentation of which the file is a part. Syntax aligned(8) class StreamingInformationBox extends FullBox('sinf') { unsigned int(32) streaming_information_flags; The following are optional fields: String MPD_location } Semantics streaming_information_flags contains the logical OR of zero or more of the following: 0x00000001 Movie Header update follows 0x00000002 Presentation Description update 0x00000004 end-of-presentation mpd_location is only present if the flags /flags Presentation Description update are active and provides a URL (uniform resource locator) for the new presentation description media. Example Use Case for MPD Updates for Live/Live Services
[000273]Let's assume that a service provider wants to provide a live football event using the extended block request streaming described here. Perhaps millions of users may want to access the event presentation. The live event is sporadically interrupted by breaks eg the break between one time and another, or another break, such as a technical stop, in the action, during which the ads/advertisements may run. Typically, there is little or no advance notice of the exact timing of interruptions.
[000274]The service provider will need a redundant infrastructure (eg encoders and servers) to enable seamless switching in case any component fails during the live event.
[000275]Let's assume that a user, named Anna, accesses the service on a bus with her mobile device and the service is available immediately. Next to her is another user, Paul, who watches the event on his laptop. A goal is scored and both celebrate this event at the same time. Paul tells Anna that the first goal of the game was even more exciting and Anna uses the service so she can watch the event 30 minutes ago. After having seen the goal, she returns to the live event.
[000276] To handle this use case, the service provider must be able to update the MPD, signal to customers that an updated MPD is available, and allow customers access to the streaming service in such a way that it can present the data in near real time.
[000277]Updating the MPD is feasible asynchronously for the distribution of segments, as described elsewhere here. The server can provide assurances to the receiver that an MPD has not been updated for some time. The server can be based on the current MPD. However, no explicit signaling is required when the MPD is updated before some minimum update period.
[000278]A fully synchronous playback is hardly achieved as the client can operate on different instances of the MPD and therefore the clients can exhibit a certain degree of “drift”. Using MPD updates, the server can transmit the changes and clients can be alerted to the changes, even during a presentation. In-band signaling on a segment-by-segment basis can be used to indicate the MPD update, so updates can be limited to segment boundaries, but this should be acceptable in most applications.
[000279]An MPD element can be added, providing the “clock time” publication time of the MPD, as well as an optional MPD update box that is added at the beginning of segments to signal that an MPD update is needed. Updates can be done hierarchically, just like with MPDs.
[000280]The “Publication Time/Instant” provides a unique identifier for the MPD and when the MPD was issued. It also provides an anchor or reference for upgrade procedures.
[000281] The MPD update box can be found in the MPD after the box/box "styp" and defined by a box/box type = "mupe", not needing any container, not mandatory and with a quantity of zero or one. The MPD update box contains information about the presentation of the media of which the segment is a part.
[000282]An example syntax is as follows: aligned(8) class MPDUpdateBox extends FullBox('mupe') { unsigned int(3) mpd information flags; unsigned int(l) new-location flag; unsigned int(28) latest_mpd_update_time; /// The following are optional fields String MPD_location } The semantics of the various objects of class MPDUpdateBox can be as follows: mpd_information_flags: the logical OR of zero or more of the following: 0x00 - media presentation description update now 0x01 - media presentation description update forward 0x02 - presentation end 0x03 to 0x07 - reserved. New_location flag: if set to 1, then the new media presentation description is available in a new location specified in mpd_location. latest_mpd_update time: Specifies the time (in ms) by when the MPD update is required relative to the latest MPD's MPD output time. The customer can choose to update the MPD at any time from now on. mpd_location: is present only if the new_location_flag is set, and if so, mpd_location provides a uniform resource locator for the new media description presentation. If the bandwidth used by an update is an issue, the server may offer MPDs for certain device features such that only those parts are updated. Time Shifted View and Network PVR
[000283]When viewing with time-shift/time shifting is supported, it may occur that the live/live session time of two or more MPDs or movie headers is valid. In this case by updating the MPD when necessary, but adding the validity mechanism or period concept, a valid MPD can exist for the entire time window. This means that the server can guarantee that any MPD and movie headers are advertised for any length of time that is within the valid time window for changing the display time. It is the customer who must ensure that their MPD and metadata available for the current time of presentation are valid. Migration from a live session to a networked PVR session using only minor MPD updates may also be supported. Special Media Segments
[000284] A problem when using the ISO/IEC 14496 12 format file within a block-on-demand streaming system is that, as described above, it can be advantageous to store the media data for a single version of the presentation in multiple files, organized in consecutive time segments. Furthermore, it may be advantageous to arrange each file with a random access point. Furthermore, it can be advantageous to choose the position of the search points during the video encoding process and segment the presentation into multiple files, each starting with a search point based on the choice of search points that was made during the encoding process. , where each random hotspot may or may not be placed at the beginning of a file, but where each file starts with a random hotspot. In a modality with the properties described above, the presentation metadata, or the media presentation description, can contain the exact duration of each file, where the duration is taken for example, to signify the difference between the start time of the media of a video file and the video media start time of the next file. Based on this information in the presentation metadata the client is able to build a mapping between the global timeline for the media presentation and the local timeline for the media within each file.
[000285]In another embodiment, the size of the presentation metadata can be advantageously reduced by specifying instead that all files or segment have the same duration. However in this case and where media files are built according to the above method, the duration of each file may not exactly equal the duration period specified in the media presentation description, because a random hotspot may not exist in the point at which is exactly the specified duration from the beginning of the file.
[000286] It will now be described another embodiment of the invention to provide a correct operation of the streaming system by requesting blocks despite the discrepancy mentioned above. In this method, an element can be provided within each file that specifies the mapping of the local media schedule within the file (by which it should be understood the schedule from timestamp zero against which the sample decoding and composition timestamps media are specified in accordance with ISO/IEC 14496-12) for the global submission schedule. This mapping information can include a single timestamp/timestamp in global presentation time that matches timestamp zero on the local file timeline. Alternate mapping information can include an offset value that specifies the difference between the global presentation time corresponding to zero hours on the local file timeline and the corresponding global presentation time for the beginning of the file according to the information provided in the presentation metadata.
[000287]Examples for these boxes/boxes may include the track fragment “decode time box” (“tfdt”) or track fragment adjustment box (“tfad”) along with the fragment media adjustment box of trail (tfma). Segment List Generation Example Adding Customers
[000288] An example customer will now be described. It can be used as a reference client to the server to ensure proper MPD generation and updates.
[000289]An HTTP streaming/streaming client is guided by the information provided in the MPD. It is assumed that the client has access to the MPD it received at time T, that is, the moment it was able to successfully receive an MPD. Determining successful reception may include the client obtaining an updated MPD, or the client verifying that the MPD has not been updated since the previous successful reception.
[000290] An example of customer behavior will be displayed. To provide a streaming service to the user, the client first analyzes the MPD and creates a list of accessible segments for each representation for the client's local time at a current system time, taking into account the list generation procedures of segments as detailed below, possibly using playlists or using URL construction rules. The client then selects one or multiple representations based on information on the representation attributes and other information, for example, client resources and available bandwidth. Depending on the grouping, representations can be presented autonomously or together with other representations.
[000291]For each representation, the client acquires the binary metadata, such as the moov header for the representation, if present, and the media segments of the selected representations. The client accesses the media content by requiring segments or segment byte ranges, possibly using the segment list. The client can initially buffer the media before starting the presentation and after the presentation starts, the client continues to consume the media content by continuously requesting segments or parts of segments taking into account the MPD update procedures.
[000292]Customer can toggle/switch representations taking into account up-to-date MPD information or up-to-date information of their environment, eg change of available bandwidth. With any request for a media segment that contains a random hotspot, the client can switch to a different representation. When moving forward, that is, the current system time (referred to as the “NOW time” to represent the relative time to the presentation), the customer consumes the accessible segments. With each advance in NOW time, the customer possibly expands the list of accessible segments for each representation according to the procedures specified in this document.
[000293] If the end of the media presentation has not yet been reached and if the current playing time approaches a limit where the client foresees the exhaustion of the media described in the MPD for the consumption of the representation, the client may request an update of MPD, with a new reception of seek time t. Once received, the client then takes into account the possibly updated MPD and the new time t in generating accessible segment lists. Figure 29 illustrates a procedure for live/live services at different times on the client. Generation of Accessible Segments List
[000294]Let's assume the HTTP streaming client has access to an MPD and may wish to generate a list of accessible segments for a NOW clock time. The client is synced to a global time reference with some precision, but advantageously no direct sync to the HTTP streaming/streaming server is required.
[000295]The list of accessible segments for each representation is preferably defined in the form of a list of pairs of segment start instants and segment locators, where the segment start time can be defined as being relative to the start of the representation without loss of generality. The start of the representation can be aligned with the start of a period or if this concept is applied. Otherwise, the start of representation can be at the start of the media presentation.
[000296]Client uses URL construction rules and timing as, for example, defined in this document. After getting a list of described segments, this list is further restricted to accessible ones, which can be a subset of the segments of the complete media presentation. Construction is governed by the current clock value at the customer's NOW time. Generally speaking, segments will only be available for any NOW time within a set of availability hours/intervals. For times outside such a window, no segments are available. Also, for live/live services, it is assumed that checktime at some point provides information about how far into the future the media is described. Checktime is defined on the time axis of the media documented in MPD; when the client's playtime reaches checktime, it advantageously requests a new MPD.
[000297]When the client's playtime reaches checktime, it advantageously requests a new MPD.
[000298]Then the list of segments is further constrained by checktime along with the MPD TimeShiftBufferDepth attribute such that only the available media segments are those for which the sum of the media segment start time and the start of the representation fall into the interval between NOW minus timeShiftBufferDepth minus the duration of the last segment described and the smallest value between checktime or NOW. Scalable Blocks
[000299]Sometimes the available bandwidth is so reduced that the block or blocks currently being received at a receiver are unlikely to be completely received in time to be played back without interrupting the performance. The receiver can detect such situations in advance. As an example, the receiver can determine that it is receiving blocks that encode 5 media units every 6 units of time and that it has an accumulation buffer of 4 media units, so the receiver can expect to have to stop or pause the presentation about 24 time units later. Early enough, the receiver can react to such a situation, for example, by abandoning the current block stream and starting to request a block or blocks of a different representation of the content, such as one that uses less amplitude/bandwidth per unit of playout/play time. As an example, if the receiver switches to a representation where blocks encoded for at least 20% more video time for blocks of the same size, the receiver might be able to eliminate the need to stop until the bandwidth situation improves. .
[000300]However, it can be wasteful for the receiver to completely discard the data already received from the representation of the abandoned representation. In one embodiment of the block streaming system described herein, the data within each block can be encoded and arranged in such a way that certain prefixes of the data within the block can be used to continue the presentation without the remainder of the block being received. As an example, the well-known scalable video encoding techniques can be used. Examples of such video encoding methods include H.264 Advanced Video Coding (AVC) - Scalable Video Coding (SVC), or the time scaling of the H.264 Advanced Video Coding (AVC) standard. Advantageously, this method allows the presentation to continue based on the part of a block that has been received, even when reception of a block or blocks may have been abandoned, for example due to changes in available bandwidth. Another advantage is that a single data file can be used as the source for many different representations of content. This is possible, for example, making use of HTTP partial GET type requests that select the subset of a block corresponding to the required representation.
[000301]A detailed improvement in this document consists of an enlarged segment, a scalable segment map. The scalable segment map contains the locations of the different layers in the segment in such a way that the client can properly access the segments' parts and extract the layers. In another modality, the media data in the segment is ordered in such a way that the segment quality increases while gradually downloading the data from the beginning of the segment. In another modality, a gradual increase in quality is applied for each block or fragment contained in the segment, such that fragment requests can be made to solve the scalable approach.
[000302]Figure 12 illustrates an aspect of scalable blocks. In this figure, a transmitter 1200 outputs metadata 1202 in scalable layer 1 (1204), scalable layer 2 (1206), and scalable layer 3 (1208), with the latter being deferred. A receiver 1210 can then use metadata 1202, scalable layer 1 (1204) and scalable layer 2 (1206) to play the media presentation 1212. Independent Staging Layers
[000303] As explained above, it is undesirable that a block request streaming system is forced to stop when the receiver is not able to receive the requested blocks of a specific representation of the requested media data in time for its reproduction, the which often creates a bad user experience. Stops can be avoided, reduced, or attenuated by restricting a representations data rate to be much less than the available bandwidth, so that it becomes very unlikely that any given part of the presentation will not be received in time , but this strategy has the disadvantage that the quality of the media is necessarily much lower than what the available bandwidth could in principle support. A lower quality presentation than what is possible can also be interpreted as a poor user experience. Thus, the designer of a streaming system for requesting blocks is faced with a choice in designing client procedures, client programming, or hardware configuration, to either request a version of content that has a much lower data rate. than the available bandwidth, in which case the user may suffer from poor media quality, or request a version of content that has a data rate close to the available bandwidth, in which case the user may suffer a high probability of pauses during presentation as available bandwidth changes.
[000304] To handle such situations, the block request streaming system described here can be configured to handle several scaling layers independently, such that a receiver can make requests in layers and a transmitter can respond to requests from layers .
[000305] In such embodiments, the encoded media data for each block may be partitioned into several separate/disconnected pieces, referred to herein as "layers", such that a combination of layers comprises the total media data of a block and such that a client that has received certain subsets of the layers can perform the decoding and rendering of a representation of the content. In this approach, the ordering of data in the stream is such that contiguous intervals increase the quality and the metadata reflects this.
[000306]An example of a technique that can be used to generate layers with the above property is the scalable video encoding technique, for example as described in the ITU T H.264/SVC standards. Another example of a technique that can be used to generate layers with the above property is the time scaling layer technique provided for in the ITU T h.264/AVC standard.
[000307] In such modalities, metadata can be provided in the MPD or in the segment itself, which allows the construction of requests for individual layers of any block and/or combinations of certain layers and/or a certain layer of multiple blocks and/or a combination of layers from multiple blocks. As an example, layers that include a block could be stored in a single file and metadata could be provided specifying byte ranges in the file corresponding to the individual layers.
[000308]A file download protocol capable of specifying byte ranges, for example HTTP 1.1, can be used to request single layers or multiple layers. Furthermore, as will be clear to those skilled in the art when reading the present description, the techniques described above referring to the construction, request and download of variable size blocks and variable combinations of blocks can also be applied in this context. combinations
[000309] Some modalities that can be advantageously employed by a block request streaming client will now be described to obtain an improvement in the user experience and/or a reduction in the capacity requirements of the server infrastructure compared to existing techniques by using tiered partitioned media data as described above.
[000310] In a first modality, the known techniques of a streaming system by requesting blocks can be applied with the modification that different versions of the content are, in some cases, replaced by different combinations of layers. That is, when an existing system can provide two distinct representations of the content in the enhanced system described herein in two layers, where one representation of the content of the existing system is similar in terms of bit rate, quality, and possibly other measures, to the first tier in the enhanced system and the second representation of the existing system's content are similar in bit rate, quality, and possibly other measures to the two-tier blend of the enhanced system. As a result the storage capacity required within the advanced system is reduced compared to that required in the existing system. Also, since existing system clients can issue requests for blocks of representation from either representation, advanced system clients can issue requests for the first or both layers of a block. As a result, the user experience on both systems is similar. In addition, improved caching is provided, as even for different qualities common threads are used which are more likely cached.
[000311] In a second embodiment, a client in an extended block-on-demand streaming system that employs the now-described layering method can maintain a separate data buffer for each of the various layers of media encoding. As will be clear to technicians in the field of data management on client devices, these “separate” buffers can be implemented by allocating regions of physically or logically separate memory to the separate buffers, or by other techniques where the buffered data is stored in a single or multiple memory regions and the separation of data from different layers is achieved logically through the use of data structures that contain references to the data storage locations coming from the separate layers and thus by the expression “buffers separate” is to be understood to include any method in which data from distinct layers can be separately identified. The client issues requests for individual layers of each block based on occupancy of each buffer eg layers can be ordered in a priority order such that a data request from one layer cannot be granted if occupancy of any buffer stops a lower layer in the order of priority is below a threshold for that lower layer. In this method priority is given to receiving data from lower layers in priority order, such that if the available bandwidth falls below that required to also receive higher layers in priority order, then only lower layers are requested. Furthermore, the thresholds associated with the different layers may be different, such that, for example, the lower layers have higher thresholds. In the case where the available bandwidth is changed such that data from a higher layer cannot be received before the block's playback/playout moment, the data for lower layers must necessarily have already been received, so the presentation can continue. with the lower layers by itself. Thresholds for buffer occupancy can be defined in terms of bytes of data, duration of replay of data contained in the buffer, number of blocks, or any other suitable measure.
[000312] In a third modality, the methods of the first and second modality can be combined in such a way that multiple media representations are provided, each including a subset of layers (as in the first modality) and such that the second modality is applied to a subset of the layers within a representation.
[000313] In a fourth modality the methods of the first, second and/or third modality can be combined with the modality in which multiple independent representations of the content are provided such that, for example, at least one of the independent representations comprises several layers so that the techniques of the first, second and/or third modalities are applied. Advanced Buffer Manager
[000314]In combination with buffer monitor 126 (see Figure 2), an advanced buffer manager can be used to optimize a client-end buffer. Block request streaming systems want to ensure that media playback can start quickly and continue smoothly while ensuring maximum media quality for the target or user device. This may require the client to request blocks that have the highest media quality, but that can also be started quickly and received on time and then played without forcing a pause in the presentation.
[000315]In modalities that use the advanced buffer manager, the manager determines which data blocks should be requested and when to make such requests. An advanced buffer manager can, for example, receive a set of metadata of the content to be presented, this metadata including a list of available representations for the content and the metadata for each representation. Metadata for a representation can include information about data rate and other parameters such as video parameters, audio or other codecs and codec parameters, video resolution, decoding complexity, audio language and any other parameters that may affect the choice of representation on the client.
[000316]The metadata for a representation may also include identifiers for the blocks into which the representation has been segmented, these identifiers providing the information necessary for the client to request a block. As an example, when the request protocol is HTTP, the identifier can be an HTTP URL, possibly with additional information identifying a byte range or time range within the file identified by the URL, such byte range or time identifying the block within the file identified by the URL.
[000317]In a specific implementation, the advanced buffer manager determines when a receiver makes a request for new blocks and can itself handle the sending of the requests. In a new aspect, the advanced buffer manager makes requests for new blocks according to the value of a balance relationship that balances high bandwidth usage and media exhaustion during a playback.
[000318]The information received by buffer monitor 126 from block buffer 125 may include indications of each event where media data is received, how much was received, when media data playout was started or stopped, and the speed of the media playback. Based on this information, buffer monitor 126 can calculate a variable that represents a current buffer size, Bcurrent. In these examples, Bcurrent represents the amount of media contained in a client or other device buffers and can be measured in time units so that Bcurrent represents the amount of time required to play all the media represented by the buffered or buffered blocks or partial blocks. buffers if no additional blocks, or partial blocks, were received. As such, Bcurrent represents the “play duration”, at normal playback speed, of media data available on the client but not yet played.
[000319]As time passes, the value of Bcurrent will decrease as the media is played and may increase each time a new block is received. Note that, for the purposes of the present description, it is assumed that a block is received when all block data is available in the block requester 124, but other measures can be used for example taking into account the reception of partial blocks . In practice, reception of a block can take place over a period of time.
[000320]Figure 13 illustrates a variation of the Bcurrent value over time as the media is played and blocks are received. As illustrated in Figure 13, the value of Bcurrent is zero for times before t0, indicating that no data was received. At t0, the first block is received and the value of Bcurrent increases to equal the playout/playback duration of the received block. At this point, playback has not yet started and so the value of Bcurrent remains constant until time t1, when a second block arrives and increases Bcurrent by the size of this second block. At this moment, the reproduction starts and the value of Bcurrent starts decreasing in a linear way, until the time t2, when a third block arrives.
[000321] The progression of Bcurrent continues like this in "sawtooth", increasing in steps each time a block is received (at times t2, t3, t4, t5 and t6) and decreasing smoothly as the data are reproduced. Note that in this example playback proceeds at the normal content playout rate and therefore the slope of the curve between block reception is exactly 1, meaning that one second of media data is played back for every 1 second of real time that elapses. . With frame-based media playing at a certain number of frames per second, eg 24 frames per second, the slant of -1 will be approximated by small step functions that indicate the playback of each individual frame of data, per example steps of 1/24 of a second when each frame is played.
[000322]Figure 14 shows another example of the evolution of Bcurrent over time. In this example, the first block arrives at t0 and playback/playout begins immediately. Block arrival and play/playout continue until time t3, at which the value of Bcurrent reaches zero. When this happens, there is no more media data available for playback, forcing the media presentation to pause. At time/instant t4, a fourth block is received and playback can resume. This example therefore shows a case where reception of the fourth block was later than desired, resulting in a pause in playback and therefore a poor user experience. Thus, it is a goal of the advanced buffer manager and other features to reduce the probability of this event while maintaining high quality media.
[000323]Buffer monitor 126 can also calculate another measure, Bratio(t), which is the ratio between the media received in a given time period and the length of the time period. More specifically, Bratio(t) is equal to Treceived/(Tnow-t), where Treceived is the amount of media (measured by its playtime) received in the time period t, some time earlier than the current time, so far, Tnow.
[000324]Bratio(t) can be used to measure the rate of change of Bcurrent. Bratio(t) = 0 is the case where no data has been received since time t; Bcurrent will be reduced by (Tnow-t) since then, assuming the media is playing. Bratio(t) = 1 is the case where media is received in the same amount that is being played, for the time (Tnow-t); Bcurrent will have the same value at time Tnow as time t. Bratio(t) > 1 is the case where more data was received than needed to reproduce in time (Tnow-t); Bcurrent will have increased from time t to time Tnow.
[000325]Buffer monitor 126 also calculates a State value, which can take a certain number of values. Buffer monitor 126 is also equipped with a function, NewState (Bcurrent, Bratio), which, given the current value of Bcurrent and Bratio values for t < Tnow, provides a new State value as output. Whenever Bcurrent and Bratio cause this function to return a value different from the current State value, the new value is assigned to State and this new State value is indicated for selector block 123.
[000326]The NewState function can be evaluated with reference to the space of all possible values of the pair (Bcurrent, Bratio(Tnow-Tx)) where Tx can be a fixed (configured) value, or can be derived from Bcurrent, for example by a configuration table that maps Bcurrent values to TX values, or it can depend on the previous value of State. Buffer monitor 126 receives one or more partitions from this space, where each partition comprises sets of disjointed regions, each region being annotated with a value of State. The evaluation of the NewState function then comprises the operation of identifying a partition and determining the region where the pair (Bcurrent, Bratio(Tnow-Tx)) resides. The return value is then the annotation associated with that region. In a simple case, a single partitioning is provided. In a more complex case, partitioning may depend on the pair (Bcurrent, Bratio(Tnow Tx)) at the time before the NewState function was evaluated or on other factors.
[000327]In a specific modality, the partitioning described above can be based on a configuration table that contains a number of threshold values for Bcurrent and a number of threshold values for Bratio. Specifically, let the threshold values for Bcurrent be Bthresh(0) = 0, Bthresh(l), ..., Bthresh(n1), Bthresh(n1+1) = ~, where n1 is the number of different threshold values from zero to Bcurrent. Let the threshold values for Bratio Br-thresh(0) = 0, thresh(l)Br, ..., Brthresh(n2), Brthresh(n2+1) = “, where n2 is the number of threshold values for Bratio. These threshold values define a partitioning comprising a grid of (nl + 1) (n2 + 1) of cells, where the i cell of the line ja corresponds to the region in which Bthresh(i 1) < Bcurrent < Bthresh(i) and r- Bthresh(j-1) <Bratio < Br-thresh(j) . Each cell in the grid described above is annotated with a State value, as it is associated with particular values stored in memory, and the NewState function below returns the State value associated with the cell indicated by the Bcurrent and Bratio(Tnow-Tx) values.
[000328]In another modality, a hysteresis value can be associated with each threshold value. In this advanced method, the evaluation of the newState function can be based on a temporary partitioning constructed using temporarily a temporarily modified set of threshold values, as follows. For each Bcurrent threshold value that is less than the Bcurrent range corresponding to the cell chosen in the last NewState evaluation, the threshold value is reduced by subtracting the hysteresis threshold value associated with that threshold. For each Bcurrent threshold value that is greater than the Bcurrent range corresponding to the cell chosen in the last NewState evaluation, the threshold value is increased by adding the hysteresis threshold value associated with that threshold. For each Bratio threshold value that is less than the Bratio range corresponding to the cell chosen in the last NewState evaluation, the threshold value is reduced by subtracting the hysteresis threshold value associated with that threshold. For each Bratio threshold value that is greater than the Bratio range corresponding to the cell chosen in the last NewState evaluation, the threshold value is increased by adding the hysteresis threshold value associated with that threshold. Modified threshold values are used to evaluate the value of NewState and then threshold values are returned to their original values.
[000329]The above description is illustrative of the basic process. As will be clear to technicians in the real-time programming field after reading this description, efficient implementations are possible. For example, each time new information is provided to buffer monitor 126, it is possible to calculate the future time NewState will change to a new value if, for example, there is no additional data to receive by the blocks. A timer/timer is then set for this time and in the absence of other inputs this timer expires which will cause the new state value to be sent to block selector 123. As a result, calculations only need to be performed when new information is supplied to the 126 buffer monitor or when a timer expires, rather than continuously.
[000330]Proper State values could be "Low", "Stable" and "Full". An example of a suitable set of threshold values and the resulting cell grid is shown in Figure 15.
[000331]In Figure 15, the Bcurrent thresholds are shown on the horizontal axis in milliseconds, with hysteresis values shown below as "+ value /". Bratio thresholds are shown on the vertical axis in permille (ie multiplied by 1000) with hysteresis values shown below as "+ value /". State values are noted in grid cells as "L", "S" and "F" for "Low", "stable" and "Full", respectively.
[000332]The block selector 123 receives notifications from the block requester 124 whenever there is an opportunity to request a new block. As described above, block selector 123 is provided with data such as the plurality of available blocks and metadata for those blocks, including for example information about the data carrier rate of each block.
[000333]The information about the media rate of a block can comprise the actual means data rate of the specific block (that is, the block size in bytes divided by the playback time in seconds), the average means rate of data of the representation to which the block belongs or a measure of the available bandwidth needed, on a sustainable basis, to perform the representation to which the block belongs without pauses, or a combination of the above.
[000334] Block selector 123 selects blocks based on the value indicated by the last State value indicated by buffer monitor 126. When this State value is "stable", block selector 123 selects a block of the same representation as the previous block selected. The selected block is the first block (in playback/playout order) containing data carriers for a period of time in the performance for which data carriers have not previously been required.
[000335]When the value of State is "Low", block selector 123 selects a block from a representation with a lower media data rate than the previously selected block. A number of factors can influence the exact choice of representation in this case. For example, block selector 123 can receive an indication of the total input data rate and can choose a representation with a data carrier rate that is less than that value.
[000336]When State value is "Full", block selector 123 selects a block from a representation with a higher media data rate than the previously selected block. A number of factors can influence the exact choice of representation in this case. For example, block selector 123 can be provided with an indication of the total input data rate and can choose a representation with a data carrier rate that is not greater than that value.
[000337]A number of additional factors can further influence the operation of block selector 123. In particular, the frequency with which the selected block data carrier rate is increased may be limited, even if monitor buffer 126 continues to indicate the "total" state. In addition, it is possible for block selector 123 to receive a "Full" state indication, but there are no higher media data rate blocks available (for example, because the last selected block was already available for the highest rate media data). In this case, block selector 123 can delay selection of the next block for a chosen time such that the total amount of buffered media data in blocking buffer 125 is delimited above.
[000338]Other factors can influence the set of blocks that are considered during the selection process. For example, the available blocks can be limited to those from representations whose encoding resolution falls within a specific range given to block selector 123.
[000339]The block selector 123 can also receive contributions from other components that monitor the other aspects of the system, such as the availability of computational resources for the decoding media. If such resources become scarce, block selector 123 can choose blocks whose decoding is indicated to be of lower computational complexity within metadata (for example, representations with lower resolution or frame rate are generally of lower decoding complexity).
[000340] The modality described above brings a substantial advantage in that using the Bratio value in the evaluation of the NewState function within the buffer monitor 126 allows a faster increase in quality at the beginning of the presentation compared to a method that only considers Bcurrent. Without considering Bratio, a large amount of buffered data can accumulate before the system is able to select blocks with higher media data rate and, consequently, higher quality. However, when the Bratio value is large, this indicates that the available bandwidth is much higher than the data support rate of the blocks previously received and that even with relatively little buffered data (ie, low value for Bcurrent), it remains safe to request blocks of the highest rate of data support and therefore the highest quality. Likewise, if the Bratio value is low (<1, for example), it indicates that the available bandwidth has dropped below the data carrier rate of the previously requested blocks, so even if Bcurrent is high, the system will change for a lower medium the data rate and consequently a lower quality, for example to avoid reaching the point where Bcurrent = 0 and the playout of the media bays. This improved behavior can be especially important in environments where network conditions and thus delivery speeds can vary quickly and dynamically, for example, users streaming to mobile devices.
[000341]Another advantage is conferred by using configuration data to specify the partitioning of the value space of (Bcurrent, Bratio). Such configuration data may be provided to buffer monitor 126 as part of the presentation metadata or by other dynamic means. Since, in practical implementations, the behavior of user network connections can be highly variable between users and over time for a single user, it can be difficult to predict a partitioning that will work well for all users. The ability to provide configuration information allows users to dynamically allow good configurations to be developed over time according to accumulated experience. Variable Request Sizing
[000342]A high order frequency may be required if each order is for a single block and if each block encodes for a short media segment. If media blocks are short, video playback is moving from block to block quickly, which provides more frequent opportunities for the receiver to adjust or change its selected data rate, changing the representation, improving the likelihood that the playout can continue without stopping. However, a disadvantage to a high frequency of requests is that they may not be sustainable on certain networks where the available bandwidth is limited on the client by the network server, for example, on wireless WAN networks such as 3G and 4G wireless WANs, where the capacity of the customer's data connection to the network is limited or may become limited for long or short periods of time due to changes in radio communication conditions.
[000343]The high frequency of requests/requests also implies a high load on the infrastructure it serves, which brings associated costs in terms of capacity requirements. Thus, it would be desirable to have some of the benefits of high order frequency without all the downsides.
[000344]In some modalities of a block request streaming system, the flexibility of the high frequency of requests is combined with the less frequent requests. In this embodiment, blocks can be constructed as described above and aggregated into segments that contain multiple blocks, also as described above. At the beginning of the presentation, the processes described above in which each block of request references from a single or multiple simultaneous requests are made to request parts of a block are applied to ensure fast channel zapping time and therefore good user experience at the beginning of the presentation. Thereafter, when a certain condition, to be described below, is met, the customer can issue orders, which cover several blocks in a single order. This is possible because blocks have been aggregated into larger files or segments and can be requested using byte or time ranges. Consecutive byte or time ranges can be aggregated into a single byte or longer time range, resulting in a single request for multiple blocks, and even discontinuous blocks can be requested in one order.
[000345] A basic configuration that can be triggered by the decision to request a single block (or a partial block) or to request several consecutive blocks is to have the configuration basing the decision on whether or not the requested blocks are likely to be reproduced or not . As an example, if it is likely that there will not be a need to switch to another representation anytime soon, then it is better for the customer to place orders for individual blocks, ie small amounts of multimedia data. One reason for this is that if a multiple block request is made when a switch to another representation may be imminent is that the switch can be made before the last few blocks of the order are played back. In this way, downloading these last blocks can delay the delivery of media data from the representation to which the switch is made, which could cause media playout stops.
[000346]However, simple block requests result in a higher frequency of requests. On the other hand, if it is unlikely that there will be a need to switch to another representation anytime soon, then it may be preferable to make multiple block requests, as all these blocks are likely to be replayable, and this results in a lower frequency of requests. , which can substantially reduce ordering overhead, especially if it is clear that there will be no imminent representation change.
[000347] In conventional block aggregation systems, the quantity ordered in each order is not dynamically adjusted, that is, typically each order is for an entire file, or each order is approximately the same quantity as the file for a representation (per times measured in time, sometimes in bytes). Thus, if all orders are smaller, then the order overhead is high, whereas if all orders are larger, then this increases the chances of media stop events and/or providing a poor quality playback service. of media if low-quality representations are chosen to avoid having to quickly switch representations as network conditions vary.
[000348]An example of a condition that, when encountered, can cause subsequent requests to reference multiple blocks, is a threshold on buffer size, Bcurrent. If Bcurrent is below the threshold, then each order issued references a single block. If Bcurrent is greater than or equal to the threshold, each request issued references multiple blocks. If an order is issued with references to multiple blocks, the number of blocks requested in each single order can be determined in one of several possible ways. For example, the number can be constant, for example two. Alternatively, the number of blocks requested in a single request can be dependent on the State of the buffer and in particular on Bcurrent. For example, a number of thresholds can be defined, with the number of blocks requested in a single request being derived from the highest of multiple thresholds which is less than Bcurrent.
[000349]Another example of a condition that, when met, can cause requests to reference multiple blocks, is the State value variable described above. For example, when State is "Stable" or "Full" then requests can be issued for multiple blocks, but when State is "Low" then all requests can be for one block.
[000350]Another mode is illustrated in Figure 16. In this mode, when the next order is to be issued (determined in step 1300), the value of current state and Bcurrent is used to determine the size of the next order. If the current State value is "Low" or the current state value is "Full" and the current representation is not the highest available (determined in step 1310, the answer is "Yes"), then the next order is chosen for be short, for example, only for the next block (block determined and requested in step 1320). The logic behind this is that these are the conditions under which it is likely that, very soon, there will be a change of representations. If the current State value is "stable" or the current state value is "Full" and the current representation is the highest available (determined in step 1310, the answer is "No"), then the duration of the consecutive blocks requested in the next order is chosen to be proportional to a fraction of α Bcurrent for some fixed α < 1 (blocks determined in step 1330, the order made in step 1340), eg for α = 0.4, if Bcurrent = 5 seconds, so the next request might be about 2 seconds worth of blocks, whereas if Bcurrent = 10 seconds, then the next request might be about 4 seconds worth of blocks. One reason for this is that, under these conditions, it may be unlikely that a change to a new representation will be made for an amount of time that is proportional to Bcurrent. Flexible Pipeline Operation
[000351]Block streaming systems can use a file request protocol that has a specific underlying transport protocol, for example TCP/IP. Starting a TCP/IP or other transport link protocol can take considerable time to reach full available bandwidth utilization. This can result in a “initialization connection penalty” each time a new connection is initiated. For example, in the case of TCP/IP, the connection initialization penalty is due to both the time taken for the initial TCP “handshake”/“handshake” to establish the connection and the time taken for the congestion control protocol to achieve full utilization of available bandwidth.
[000352]In this case, it may be desirable to issue multiple requests, using a single connection, in order to reduce the frequency with which the initialization connection penalty is constituted. However, some file transport protocols, for example HTTP, do not provide a mechanism to cancel a request other than to close the transport layer connection at all and thus incur an initialization connection penalty when a new connection is established in place of the old one. An issued request may need to be canceled if it is determined that the available bandwidth has changed and a different media data rate is needed instead, ie there is a decision to switch to a different representation. Another reason for canceling an order might be issued if the user has requested that the media presentation be ended and a new presentation started (perhaps from the same content item at a different point in the presentation, or perhaps from a new content item) .
[000353] As is known, the connection initialization penalty can be avoided by keeping the connection/connection open and reverting to using the same connection for subsequent requests and, as is also known, the connection can be kept fully used if multiple requests are issued at the same time over the same connection/connection (a technique known as “pipelining” in the context of HTTP). However, a disadvantage of issuing multiple requests at the same time, or, more generally, such that multiple requests are issued before previous requests have been completed over a link, may be that the link is then compromised to the execution of the response to these requests and then, if changes to which requests should be issued become desirable, the connection can be closed if it becomes necessary to cancel already issued requests that are no longer desired.
[000354] The probability that an issued order needs to be canceled may be, in part, dependent on the length of the time interval between the issuing of the order and the playing time of the requested block in the sense that when this time interval is high, the probability that an issued order needs to be canceled is also high (because the available bandwidth is likely to change during the interval).
[000355]As is well known, some file download protocols have the property that a connection to a single underlying transport layer can be advantageously used for multiple transfer requests. For example, HTTP has this property, since reusing a single connection for multiple requests avoids the initialization connection penalty described above for requests other than the first. However, a disadvantage of this approach is that the connection is committed to carrying the requested data in each order issued and therefore if an order or orders have to be canceled then the connection can be closed, incurring the connection penalty of initialization when a replacement connection is established, or the client can wait to receive data that is no longer needed, incurring a delay in receiving subsequent data.
[000356] A modality will now be described which retains the advantages of connection reuse without incurring this disadvantage and which also increases the frequency with which connections can be reused.
[000357]The modalities of block request streaming systems described here are configured to reuse a connection for several requests without having to compromise the connection at the beginning of a certain set of requests. Essentially, a new request is issued on an existing connection when the requests already issued on the connection have not yet completed but are nearing completion. One reason not to wait until existing requests complete is that if previous requests complete, then the connection speed could degrade, ie the underlying TCP session could go into an idle state, or the TCP variable cwnd could be substantially reduced, thereby substantially reducing the initial download speed of the new order issued on that connection. One reason to wait until near completion before issuing an additional order is because if a new order is issued long before the previous completed orders, then the newly issued order may not even start for some substantial period of time, and that could be the case that during this period of time before the newly issued order begins the decision to place the new order is no longer valid, for example, due to a decision to change representations. Thus, embedding clients that implement this technique will issue a new request on a connection as late as possible without slowing down the transfer capabilities of the connection.
[000358]The method comprises controlling the number of bytes received from a connection, in response to the last request issued on this connection and applying a test for that number. This can be done by getting the receiver (or transmitter, if applicable) configured to monitor and test.
[000359] If the test “pass” ie passes, then a new order can be issued on connection. An example of a suitable test is whether the number of bytes received is greater than a fixed fraction of the size of the requested data. For example, this fraction could be 80%. Another example of a suitable assay is based on the following calculation, as illustrated in Figure 17. In the calculation, let R be an estimate of the binding data rate, T an estimate of the round-trip time ("RTT"), and X the numerical factor which, for example, could be a set of constants for a value between 0.5 and 2, where the estimates of R and T are updated on a regular basis (updated in step 1410). Let S be the size of the data required in the last request, B is the number of bytes of the required data received (calculated in step 1420).
[000360]A suitable test would be to have the receiver (or transmitter, if applicable) run a routine to evaluate the inequality (SB) <X • R • T (tstate in step 1430), and if the answer is "Yes", take an action. For example, a test could be done to see if there is another request ready to be issued on the connection (tstate in step 1440), and if it is "Yes", then issue the request for the connection (step 1450) and if " No" then the process returns to step 1410 to continue updating and testing. If the test result in step 1430 is “no” then the process returns to step 1410 to continue updating and testing.
[000361] The inequality test in step 1430 (performed by properly programmed elements, for example) causes each subsequent request to be issued when the amount of data remaining to be received is equal to X times the amount of data that can be received at the current reception rate estimated at an RTT (round trip time). A number of methods for estimating the data rate, R, at step 1410 are known to those skilled in the art. As an example, the data rate can be estimated as DT/t, where Dt is the number of bits received in the preceding seconds t and where t can be, for example, 1 s or 0.5 s or some other interval. Another method is an exponential weighted average, or infinite impulse response (IIR) first-order filter average of the input data rate. A number of methods for estimating the RTT,T, at step 1410 are known to those skilled in the art.
[000362]The test in step 1430 can be applied to the total of all active links over an interface, as explained in more detail later.
[000363] The method also includes the construction of a list of requests/candidate requests, associating each candidate request with a set of suitable servers so that the request can be made and ordering the list of candidate requests in order of priority. Some entries in the candidate request list may have the same priority. Servers in the list of suitable servers associated with each candidate request are identified by “host” names. Each hostname corresponds to a set of Internet Protocol addresses that can be obtained from the DNS - Domain Name System (Domain Name System) as it is well known. Therefore, each possible request in the candidate request list is associated with a set of Internet Protocol addresses, specifically the union of the sets of IP (Internet Protocol) addresses associated with the hostnames associated with the servers associated with the request. candidate. Whenever the test described in step 1430 is met for a connection, and no new request has yet been issued on that connection, the priority request in the candidate request lists with which the Internet Protocol address of the connection's destination is associated is chosen. , and this request is issued on connection. The order is also removed from the candidate order list.
[000364]Candidate orders can be removed (cancelled) from the candidate list, new orders can be added to the candidate list with a higher priority than existing orders on the candidate list, and existing orders on the candidate list may have their priority changed. The dynamic nature of which orders are in the candidate list, and the dynamic nature of their priority in the candidate list, which may change orders may be issued next depend on when a test of the type described in step 1430 is passed.
[000365] As an example, it could be possible that if the answer to the test described in step 1430 is "Yes" at some time t, then the next request would be issued an A request, whereas if the answer to the test described in step 1430 is not "Yes" until some time t'>t, so the next order issued would become a B order, because either an order was removed from the candidate order list between times t'>t', or because a B order was added to the candidate list with higher priority than an order of time between t and t', or because an order B was inserted into the candidate list at time t, but with lower priority than an order A, and between time tet' the priority of request B has been made higher than the request of A.
[000366]Figure 18 illustrates an example of a wish list in the wish candidate list. In this example, there are three links, and there are six requests in the candidate list, identified as A, B, C, D, E, and F. Each of the requests in the candidate list can be issued with a subset of the links as indicated, the request, for example, A can be issued in connection 1, while request F can be issued in connection 2 or a connection 3. The priority of each request is also marked in fig. 18, and a lower priority value indicates that a request is higher priority. Thus, orders A and B with a priority of 0 are the highest priority requests, while orders F with a priority value of 3 is the lowest priority among the orders in the candidate list.
[000367] If, at this moment, at time t, connection 1 passes the test described in step 1430, request A or B is issued on connection 1. If instead connection 3 passes the test described in step 1430 in this time t, then D is issued on connection 3, since D is the highest priority request that can be issued on connection 3.
[000368] Let's assume that for all connections the response to the test described in step 1430 from time t to some time after t' is "No", and between time tet' the request changes its priority from 0 to 5, the order B is removed from the candidate list, and a new order G with a priority of 0 is added to the candidate list. Then, at time t', the list of new candidates can be as shown in Figure 19.
[000369]If at time t' connection 1 passes the test described in step 1430, then request C with priority 4 is issued on connection 1, as it is the highest priority request in the list of candidates that can be issued on the connection 1 at this point in time.
[000370]Let's assume in the same situation that instead request A would have been issued on connection 1 at time t (which was one of the two highest priority choices for connection 1 at time t, as shown in Figure 18) . Since the response to the test described in step 1430 from time t to some time after t' is “No” for all connections, connection 1 is still providing data until at least time t' for requests issued before time te, therefore, request A would not have started at least until time t'. Issuing request C at time t' is a better decision than issuing request A at time t, since request C starts at the same time after t' that request A would have started, and given that at this point the request C is of higher priority than order A.
[000371] As another alternative, if the type test described in step 1430 is applied to the aggregate of active links a link can be chosen having a destination whose IP address is associated with the first request in the candidate request list or another request with the same priority as the first order.
[000372]Several methods are possible for building the candidate order list. For example, the candidate list may contain n orders representing orders for the next n pieces of data from the current representation of the presentation in time sequence order, where the order for the first piece of data has the highest priority and the order for the last piece of data that has lower priority. In some cases it may not be one. The value of n can depend on the buffer size, Bcurrent, or the state variable or another measure of the client's buffer occupancy state. For example, a number of threshold values can be defined for Bcurrent and a value associated with each threshold, then the value of n is considered to be the value associated with the largest threshold that is less than Bcurrent.
[000373] The modality described above guarantees flexible allocation of connection requests, ensuring that preference is given to reusing an existing connection, even if the priority request is not suitable for that connection (because the destination IP address of the connection is not one that is assigned to any of the hostnames associated with the request). Dependence on n of Bcurrent or State or another measure of client buffer occupancy ensures that such requests/requests are "out of priority order", such requests are not issued when the client has an urgent need to issue and complete the order associated with next portion of data to be played back in time sequence.
[000374]These methods can be advantageously combined with cooperative HTTP and FEC. Consistent Server Selection
[000375]As is well known, files to be downloaded using a file download protocol are usually identified by an identifier comprising a host and a filename. For example, this is the case for the HTTP protocol, in which case the identifier is a uniform resource identifier (URI). A hostname can correspond to multiple hosts identified by Internet Protocol addresses. For example, this is a common method of spreading the order load from multiple customers across multiple physical machines. In particular, this approach is typically taken by “Content Delivery” (CDN) networks. In this case, a request issued in connection with any of the physical hosts/hosts should be successful. A number of methods are known by which a client can select from Internet Protocol addresses associated with a host. For example, these addresses are typically provided to the customer through the Domain Name System and are provided in order of priority. A client can then choose the highest priority (first) of Internet protocol addresses. However, there is generally no coordination between clients on how this choice is made, with the result that different clients may request the same file from different servers. This can result in the same file being cached on multiple servers nearby, which reduces the efficiency of the caching infrastructure.
[000376] This can be managed by a system that advantageously increases the probability that two clients requesting the same block will request this block from the same server. The new method described here comprises selecting among the available Internet Protocol addresses in a way determined by the identifier of the file to be requested and in such a way that different clients are presented with the same or similar choices of Internet Protocol addresses and file identifiers will make the same choice.
[000377]A first modality of the method is described with reference to Figure 20. The client first obtains a set of Internet Protocol IP1, IP2, ..., IPn as shown in step 1710. If there is a file for which requests must be issued, as decided in step 1720, the client determines which Internet Protocol address to issue requests for the file, as determined in steps 1730 to 1770. Given a set of Internet Protocol addresses and an identifier for a file to be requested the method comprises ordering the Internet Protocol in a way determined by the file identifier. For example, for each Internet Protocol address a sequence of bytes is constructed comprising the concatenation of the Internet Protocol address and the file identifier as shown in step 1730. A hash function is applied to this byte string as shown in step 1740, and the resulting hash values are arranged according to a fixed ordering, as shown in step 1750, for example increasing the order number, inducing an ordering on the Internet Protocol addresses. The same hash function can be used by all clients, thus ensuring that the same result is produced by the hash function on a given input by all clients. The hash function can be statically configured for all clients in a set of clients, or all clients in a set of clients can get a partial or full description of the hash function when clients get the list of Internet Protocol addresses, or all clients in a client set can get a partial or full description of the hash function when clients get the file handle, or the hash function can be determined by other means. The Internet Protocol address that is first in this order is chosen and this address is then used to establish a request and send connection for all or parts of the file, as shown in steps 1760 and 1770.
[000378]The above method can be applied when a new connection is established to request a file. It can also be applied when a number of established connections are available and one of them can be chosen to issue a new order.
[000379] Furthermore, when an established link is available and an order can be chosen from a set of candidate requests with equal priority an ordering on the candidate requests is induced, for example, by the same method of hash values above and the candidate order that appears first in this ordering is chosen. The methods can be combined to select both a connection and candidate request from a set of equal priority bindings and requests, again computing a hash for each connection and request combination, ordering these hash values in an order fixing and choosing the combination that occurs first in the ordering induced on the set of combinations of requests and connections.
[000380]This method has advantages for the following reason: a typical approach taken by a block server infrastructure such as that shown in Figure 1 (BSI 101) or Figure 2 (BSIs 101), and in particular an approach commonly taken by CDNs , is to provide multiple caching proxy servers that receive requests from clients. A proxy server cache cannot be provided with the requested file in a given request and, in this case, such servers typically forward the request to another server, receive the response from that server, typically including the requested file, and forward the response to the client . The cache proxy server can also store (cache) the requested file so that it can immediately respond to subsequent requests for the file. The common approach described above has the property that the set of files stored in a particular cache proxy server is largely determined by the set of requests that the cache proxy server has received.
[000381]The method described above has the following advantage. If all clients in a set of clients receive the same list of Internet Protocol addresses, then these clients will use the same Internet Protocol address for all requests issued to the same file. If there are two different Internet Protocol address lists and each client receives one of these two lists, the clients will use a maximum of two different Internet Protocol addresses for all requests issued to the same file. In general, if the Internet Protocol address lists provided to clients are similar, the clients will use a small set of Internet Protocol addresses provided for all requests issued to the same file. Because nearby clients tend to receive similar lists of Internet Protocol addresses, it is likely that upcoming requests from clients target a file from only a small portion of the proxy cache servers available to those clients. Thus, there will be only a small fraction of the proxy server cache than the file cache, which advantageously minimizes the amount of cache resources used to store the file.
[000382] Preferably, the hash function has the property that a very small fraction of different inputs are mapped to the same output, and that different inputs are mapped to essentially random outputs, to ensure that for a given set of addresses. Internet Protocol, the proportion of files for which a given one of the Internet Protocol addresses is first in the sorted list produced by step 1750 is approximately the same for all Internet Protocol addresses in the list. On the other hand, it is important that the hash function is deterministic, in the sense that for a given input the output of the hash function is the same for all clients.
[000383]Another advantage of the method described above is the following. Let's assume that all clients in a client set are provided with the same Internet Protocol address list. Because of the properties of the hash function just described, it is likely that requests for different files from these clients will be evenly distributed across the entire Internet Protocol address pool, which in turn means that requests will be evenly distributed across the cache. proxy servers. In this way, the cache resources used to store these files are evenly distributed among the cache proxy servers, and file requests are evenly distributed among the cache proxy servers. Thus, the method provides both storage balancing and load balancing across the caching infrastructure.
[000384]Several variations to the approach described above are known to those skilled in the art, and in many cases these variations retain the property that the set of files stored in a proxy is determined at least in part by the set of requests to the proxy server cache received. In the common case where a given hostname resolves to multiple physical cache proxy servers, it will be common that all of these servers will eventually store a copy of any file as it is frequently requested. This duplication may be undesirable, as the storage resources on proxy cache servers are limited and as result files may, on occasion, be removed (purged) from the cache. The new method described here ensures that requests for a data file are directed to the proxy server cache in such a way that this duplication is reduced, thus reducing the need to remove files from the cache and thus increasing the probability that any data file is present (that is, not purged from) in proxy cache memory.
[000385]When a file is present in the proxy cache, the response sent to the client is faster, which has the advantage of reducing the probability that the requested file arrives late, which can result in a pause in the playback media and therefore a poor user experience. Furthermore, when a file is not present in the proxy cache the request can be sent to another server, causing additional load on both the server infrastructure and the network connections between the servers. In many cases, the server to which the request is sent may be in a distant location and transmitting the file from this server to the proxy cache server may incur transmission costs. Therefore, the new method described here results in a reduction of such transportation costs. Probabilistic Requests for Complete Files
[000386]A particular concern in the case where the HTTP protocol is used with interval requests is the behavior of cache servers that are commonly used to provide scaling of the server infrastructure. While it may be common for HTTP cache servers to support the HTTP banner header, the exact behavior of different HTTP cache servers varies by implementation. Most cache server implementations fulfill cache range requests in the case that the file is available in the cache. A common implementation of HTTP Cache servers always forwards HTTP requests downstream containing interval header to an upstream node, unless the cache server has a copy of the file (cache server or origin server). In some implementations the upstream Request Range response is the entire file, and this file is cached and the Range request response is extracted from this file and sent. However, in at least one implementation the upstream response to the Range request is just the data bytes in the range request itself, and these data bytes are not stored but rather simply sent as the response to the range request a downstream. As a result, the use of Range headers by clients can have the consequence that the file itself is never cached and the desirable network scaling properties will be lost.
[000387]In the above, the operation of proxy cache servers was described, as well as the method of requesting Blocks from a file that is an aggregation of several blocks. As an example, this can be achieved through the use of the HTTP banner request header. Such requests are called “partial orders” in the following. Another embodiment will now be described, which has advantage in the case where the block infrastructure 101 does not provide full support for the HTTP range/slot header. Commonly, servers within a block server infrastructure, for example a Content Delivery Network, support partial requests, but cannot store the response to partial requests for local storage (cache). That server can fulfill a partial request by forwarding the request to another server, unless the entire file is stored in local memory, in which case the response can be sent without forwarding the request to another server.
[000388] A block request streaming system that makes use of the block aggregation enhancement described above may run poorly if the block server infrastructure exhibits this behavior, because all requests, being partial requests, will be forwarded to another server and no requests will be served by cache proxy servers, making it impossible for the object to provide cache proxy servers in the first place. During the block request streaming system process as described above, a client may at some point request a block that is at the beginning of a file.
[000389]According to the new method described here, whenever a certain condition is met, such requests can be converted from requests for the first block into a file-wide requests file. When a request for the entire file is received by a proxy server the proxy server's cache usually stores the response. Therefore, the use of these requests causes the file to be brought into the cache from the local proxy cache servers that subsequent requests, either for the full file or partial requests can be served directly by the cache proxy server. The condition can be such that, among a set of requests associated with a given file, for example the set of requests generated by a set of customers viewing the content item in question, the condition will be satisfied by at least a fraction of these requests. .
[000390]An example of a suitable condition is that a randomly chosen number is above a given threshold. This threshold can be set such that the conversion of a single block order into a full file order occurs, on average, for a provided fraction of the orders, for example once in ten (in which case the random number can be chosen from the range [0, 1] and the limit can be 0.9). Another example of a suitable condition is that a hash function computed over some information associated with the block and some information associated with the client has one of a given set of values. This method has the advantage that for a file that is frequently requested, the process will be taken to the cache of a local proxy server, however the functioning of the block request transmission system is not significantly changed from the default operation in which each request is for a single block. In many cases where order conversion from a single block request to a whole file request takes place, the client procedures would otherwise go to request the other blocks within the file. If this is the case then these requests can be suppressed because the blocks in question will be received in any case as a result of the complete file request. URL Construction and Segment List Generation and Search
[000391] Segment list generation deals with the question of how a customer can generate a segment list from MPD at a NOW moment of customer local time for a specific representation that starts at some starttime/start time , either in relation to the beginning of the presentation of the means for cases on demand or expressed in clock time. A segment list may comprise a locator for example of a URL for an initial optional metadata representation, as well as a list of media segments. Each media segment can be assigned a starttime, duration, and locator. Starttime typically expresses an approximation of the media time of the media contained in a segment, but not necessarily an exact sample time. The starttime is used by the HTTP streaming client to issue the download request at the appropriate time. The segment list generation, including the start time of each one, can be done in different ways. The URLs can be provided as a reading list or a URL construction rule can be advantageously used for a compact representation of the segment list.
[000392]A list of segments based on URL construction can, for example, be performed if the MPD signals it for a specific attribute or element, such as FileDynamicInfo or an equivalent signal. The generic way to create a list of segments of a URL construct is provided below in the “URL construct - Overview” section. A playlist/playlist based build can, for example, be signaled by a different signal. Searching the segment list and getting an accurate media time/instant is also advantageously implemented in this context. URL Construction - Overview
[000393] As described above, in an embodiment of the present invention a metadata file containing build URL rules that allow client devices to build file identifiers for presentation blocks can be provided. A new enhancement to the block-on-demand streaming system will now be described that provides for changes to the metadata file, including changes to URL construction rules, changes to the number of available encodings, changes to metadata associated with the available encodings, such such as bitrate, aspect ratio, resolution of audio, video or codec or codec parameters and other parameters.
[000394] In this new enhancement additional data associated with each element of the metadata file can be provided indicating a time interval within the global presentation. Within this time frame the element can be considered valid and otherwise the time frame the element can be ignored. In addition, the metadata syntax can be augmented so that elements previously allowed to appear only once or at most once can appear multiple times. An additional restriction may apply in the present case, which provides that for such elements the specified time intervals must be separate. At any given time instant, considering only those elements whose interval time contains instant given time results in a metadata file that is consistent with the original metadata syntax. We call time intervals such validity intervals. This method therefore provides for flagging within some single file metadata changes of the type described above. Advantageously, such a method can be used to provide a media presentation that supports changes of the type described at specific points within the presentation. URL Builder
[000395]As described herein, a common feature of block request streaming systems is the need to provide the client with "metadata" that identify the available media encodings and provide the information necessary for the client to request blocks of these encodings. For example, in the case of HTTP, this information can include URLs to files containing the media blocks. A playlist file can be provided that lists the URLs for blocks of a given encoding. Several playlist files are provided, one for each encoding, along with a playlist master list that lists the playlists corresponding to the different encodings. A disadvantage of this system is that the metadata can become very large and therefore takes some time to be requested when the client starts transmitting. Another disadvantage of this system is evident, in the case of live content, when the files corresponding to the blocks of data carriers are generated "on the fly"/"on-the-go" from a media stream that is being captured in real time (live), for example a live sporting events or news program. In this case, list files can be refreshed every time a new block is available (eg every few seconds). Client devices can repeatedly search the playlist file to determine if new blocks are available and get their URL. This can place a significant burden on the infrastructure it serves and on particular means that metadata files cannot be stored longer than the update interval, which is equal to the block size which is commonly on the order of a few seconds.
[000396] An important aspect of a transmission block request system is the method used to inform clients of file identifiers, eg URLs, which must be used, in conjunction with the file download protocol, to request blocks . For example, a method where for each representation of a presentation a list file is provided that lists the URLs of the files containing the multimedia data blocks. A disadvantage of this method is that at least some of the playlist file itself needs to be downloaded before playback can begin, increasing channel zapping time and therefore causing a poor user experience. For a long media presentation with multiple representations or many, the list of file URLs can be large and therefore the list file can be large further increasing channel zapping time.
[000397]Another disadvantage of this method occurs in case of live content. In this case, the complete list of URLs is not made available in advance and the playlist file is periodically updated as new blocks become available and clients periodically demand the playlist file in order to receive the updated version. Because this file is updated frequently, it cannot be stored for a long time inside the cache proxy servers. This means that many of the requests for this file will be sent to other servers and eventually to the server that generates the file. In the case of a popular media presentation this can result in a high load on the server and network, which can in turn result in a slow response time and therefore a high channel zapping time and a poor experience for the user. In the worst case, the server becomes overloaded and this results in some users being unable to see the presentation.
[000398] It is desirable in designing a block-on-demand streaming system to avoid restrictions placed on the form of file identifiers that can be used. This is because a number of considerations can motivate the use of identifiers in a particular way. For example, in the case where the block server infrastructure is a content distribution network, there may be file naming or storage conventions related to the desire to distribute storage or serve the load across the network or other requirements that lead to to certain forms of file identifier that cannot be predicted at the time of system design.
[000399]Another modality will now be described, which alleviates the aforementioned drawbacks, while retaining the flexibility to choose the appropriate file identification conventions. In this method metadata can be provided for each representation of the media presentation that includes an identifier construction rules file. The identifier construction rules file can, for example, comprise a text. In order to determine the file identifier for a given block of presentation, a method of interpreting the identifier file construction rule can be provided, this method comprising determining input parameters and evaluating the identification file construction rule along with the input parameters. Input parameters can, for example, include an index of the file to be identified, where the first file has index zero, the second has an index, the third has index two, and so on. For example, in the case where all files span the same time period (or roughly the same time period), then the file index associated with any given time within the presentation can be easily determined. Alternatively, the submission deadline covered by each file can be provided within the submission or in the version metadata.
[000400]In one embodiment, the file identifier construction rule may comprise a string or text string that may contain certain special identifiers corresponding to the input parameters. The file identifier construction rule evaluation method comprises determining the positions of the special identifiers within the text string and replacing each such special identifier with a string representation of the corresponding input parameter value.
[000401]In another embodiment, the file identifier construction rule may comprise a text string conforming to an expression language. An expression language comprises the definition of a syntax so that expressions in the language can conform and a set of rules for evaluating a string according to the syntax.
[000402] A specific example will now be described with reference to Figure 21 et seq. An example of a syntax definition for a suitable expression language, defined in Extended Backus-Naur Form, is as shown in Figure 21. An example of rules for evaluating a string conforming to the output <expression> in Figure 21 comprises transform recursively the conformant string for output <expression> (an <expression>) into a conformant string for output <literal> as follows: A conformant <expression> for output <literal> remains unchanged. A conformant <expression> for output <variable> is replaced with the value of the variable identified by the string <token> for output <variable>. A conformant <expression> for the <function> output is evaluated by evaluating each of its arguments according to these rules and applying a transformation to those arguments dependent on the <token> element of the <function> output as described below. A conformant <expression> for the last alternative of the <expression> production is evaluated by evaluating the two <Expression> elements and applying an operation to these arguments dependent on the <operator> element of the last alternative of the <expression> production as described below .
[000403] In the method described above it is assumed that the evaluation is carried out in a context in which a plurality of variables can be defined. A variable is a (name, value) pair, where "name" is a conformant string for output <token> and "value" is a conformant string for output <literal>. Some variables can be defined outside of the assessment process before the assessment begins. Other variables can be defined within the assessment process itself. All variables are "global" in the sense that only one variable exists with every possible "name".
[000404]An example of a function is the "printf" function. This function accepts one or more arguments. The first argument can be conformant to the output <string> (hereafter a "string"). The printf function evaluates to a transformed version of its first argument. The transformation used is the same as the "printf" function from the standard C library, with the additional arguments included in the output <function> providing the additional arguments up to the standard C library function printf.
[000405]Another example of a function is the "hash" function. This function accepts two arguments, the first of which can be a string and the second of which can conform to the output <number> (hereafter a “number”). The "hash" function applies a hash algorithm to the first argument and returns a result that is a non-negative integer less than the second argument. An example of a hash function is suitable given in function C shown in fig. 22, whose arguments are the input string (excluding the enclosing quotes) and the numeric input value. Other examples of hash functions are well known to those skilled in the art.
[000406]Another example of a function is the "subst"function which has string arguments of one, two or three. In the case where an argument is given the result of the "Subst" function is the first argument. In the case where two arguments are given, the result of the "Subst" function is calculated by eliminating all occurrences of the second argument (excluding the enclosing quotes) in the first argument and returns the first argument as modified. In the case where three arguments are given, the result of the "Subst" function is calculated by replacing all occurrences of the second argument (excluding the quotes enclosing) within the first argument with the third argument (excluding the quotes enclosing) and returning the first argument so modified.
[000407]Some examples of operators are the addition, subtraction, division, multiplication and modulo operators, identified by the productions <operator>'+', '', '/', '*', '%', respectively. These carriers require the <Expression> productions each side of the <operator> production to evaluate in numbers. Operator evaluation comprises applying the appropriate arithmetic operation (addition, subtraction, division, multiplication, and modulus, respectively) to these two numbers in the usual way and returning the result in a way compatible with the production <number>.
[000408]Another example of an operator is the assignment operator, identified by the output <operator>'='. This operator requires that the left argument evaluates to a string whose content is compatible with the production <token>. The content of a string is defined as the string of characters enclosed in quotation marks. The equality operator causes the variable whose name is the <token> equal to the content of the left argument to be assigned a value equal to the result of evaluating the right-hand argument. This value is also the result of evaluating the expression of the operator.
[000409]Another example of an operator is the sequence operator, identified by the <operator>production ';'. The result of evaluating this operator is the right-hand argument. Note that, as with all operators, both arguments are evaluated and the left argument is evaluated first.
[000410] In an embodiment of the present invention, the identifier of a file can be obtained by evaluating an identifier construction rule file according to the above rule with a specific set of input variables that identify the intended file. An example of an input variable is the variable with the name "index" and equal value to the numeric index of the file within the presentation. Another example of an input variable is the variable with the name "bitrate" and the value equal to the average bitrate of the required version of the presentation.
[000411]Figure 23 illustrates some examples of construction identifier file rules, where the input variables are "id", giving an identifier for the representation of the desired presentation and "seq", giving a sequence number for the file.
[000412]As will be clear to technicians in the field after reading this description, numerous variations of the above method are possible. For example, not all functions and operators described above can be provided or additional functions or operators can be provided. URL Construction and Timing Rules
[000413]This section provides basic URI construction rules for assigning a file or segment URI, as well as a start time for each segment within a media representation and presentation.
[000414]For this clause the availability of a media presentation description on the client is assumed.
[000415]Let's assume the HTTP streaming client is playing off media that is downloaded inside a multimedia presentation. Actual presentation HTTP client time can be defined as where the presentation time is relative to the start of the presentation. At startup, time t=0 presentation can be assumed.
[000416]At any point t, the HTTP client can download any data with TP playback time (also relative to the start of the presentation) at most, ahead of MaximumClientPreBufferTime t actual time presentation and the data that is needed due to a user interaction, eg seek, forward, etc. In some embodiments the MaximumClientPreBufferTime cannot even be specified in the sense that a client can transfer data ahead of the current game time tP without restrictions.
[000417]The HTTP client can avoid unnecessary data downloading, for example all the segments of representations that are not expected to be thrown away cannot normally be downloaded.
[000418] The basic process in providing streaming services can be data downloading by generating appropriate requests to download entire files/segments or subset of files and segments, for example, using HTTP GET requests or partial HTTP GET requests . This description deals with how to access data for a specific game tP time, but generally the client can download data for a larger time interval than game time to avoid inefficient requests. The HTTP client can minimize the number/frequency of HTTP requests in providing the streaming service.
[000419]To access media data at playtime tP or at least close to tP in a specific representation, the client determines the URL for the file containing that playback time and further determines the range of bytes in the file to access this playback instant.
[000420]The media presentation description can assign a representation id, r, to each representation, for example by using the RepresentationID attribute. In other words, the MPD content, when written by the ingest system or when read by the client, will be interpreted such that there is an attribution. To transfer data for a specific playtime tP to a specific representation with identification r, the client can construct an appropriate URI for a file.
[000421]The media presentation description can assign to each file or segment of each representation r the following attributes: (a) a sequence number i within the representation file r, with i = 1, 2,..., Nr; (b) the relative start time of the file with representation of ID r and file index i relative to the presentation time, defined as ts(r,i); (c) the file URI for the segment/file with index of r and representation identification file, denoted as FileURI (r, i).
[000422]In one modality the start time of the file and the URIs of the files can be provided explicitly for a representation. In another embodiment, a list of file URIs can be provided, where each file URI explicitly gets index i according to position in the list and the segment start time is derived as the sum of all segment durations for the segments 1 to i-1. The duration of each segment can be given according to any of the rules discussed above. For example, anyone with knowledge in the area of basic math can use other methods to derive a methodology to easily derive the start time/instant of a single element or attribute and the index URI file position in the representation.
[000423] If a dynamic build rule is provided in the MPD, then the start time of each file and each file URI can be built dynamically by using a build rule, the requested file index and potentially some additional parameters provided in the media presentation description. Information can be provided for example in MPD attributes and elements such as FileURIPattem and FileInfoDynamic. The FileURIPattem provides information on how to construct URIs based on the file sequence number of index i and the representation ID r. The FileURIFormat is calculated by: FileURIFormat = sprintf ("%s%s%s%s%s.%s", BaseURI, BaseFileName, RepresentationIDFormat, SeparatorFormat, FileSequenceIDFormat, FileExtension); and the FileURI(r,i) is constructed by: FileURI(r,i) = sprintf (FileURIFormat, r, i);
[000424]The relative start time ts(r,i) for each segment/file can be derived by some attribute contained in the MPD describing the duration of the segments in this representation, for example, the attribute FileInfoDynamic. The MPD can also contain a sequence of FileInfoDynamic attributes that is global to all representations of the media presentation or at least to all representations in a period in the same way as specified above. If media data for a specific tP, playtime, in the r representation, is requested, the corresponding index i(r,tP) can be derived as i(r,tp) such that the playtime of this index is in the range between the start time of ts(r, i(re tP)) and ts(r, i(re tP) + 1). Segment access can be further restricted for the above cases, for example the segment is not accessible.
[000425]To access the exact playback time tP, once the corresponding segment index and URI are obtained, depends on the actual segment format. This example assumes that media segments have a local time line that starts at 0 without loss of generality. To access and present the data in playback time/instant tP, the client can download the corresponding data for the local time of the file/segment which can be accessed through the URI FileURI(r,i) with i = i(r, tp ).
[000426]In general, customers can download the entire file and can then access tP - instant playback. However, not necessarily all information in the 3GP file is required for download, as the 3GP file provides structures for mapping local timing to byte ranges. Therefore, only specific byte ranges to access tP - play time - may be sufficient to play the media, provided that sufficient random access information is available. Sufficient information about structure and mapping of the byte range and the local timing of the media segment at the beginning of the segment can also be provided, for example using a segment index. Having access to the beginning, for example 1200 bytes, of the segment, the client can have enough information to directly access the required byte range for the TP playback instant.
[000427]Another example assumes that the segment index, possibly specified as the box/box “tidx” as below, can be used to identify the byte offsets/offsets of the required fragment or fragments. Partial GET requests can be formed for the desired fragment or fragments. There are other alternatives, for example the client can issue a default request for the file and cancel this when it receives the first "tidx" box. Search/Search
[000428]A client can try to look for a specific presentation time tp in a representation. Based on the MPD, the client has access to the media segment start time/time and the URL of each media segment in the representation. The client can get the segment index, segment_index of the segment most likely to contain media samples for the presentation time tp as the maximum segment index i for which the start time tS(r,i) is smaller or equal to the presentation time tp ie segment_index = max{iltS(r,i) <tp}. The segment URL is taken as FileURI(r,i).
[000429]Note that timing/timing information in the MPD can be approximate due to issues related to placement of random access points, media strip alignment and media timing drift. As a result, the segment identified by the above procedure might start at a time just after tp and the media data for the presentation time tp might be in the previous media segment. In the case of search, the search time can be updated to match the instant of the first sample of the retrieved file, or the previous file that can be retrieved instead. However, note that during continuous playback, including cases where there is a switch between alternate representations/versions, media data for the time between tp and the start of the retrieved segment will be available.
[000430]For accurate search of a tp presentation instant, the HTTP streaming client needs to access a random access point (RAP). To determine the random access point in a 3GPP adaptive HTTP streaming media segment, the client can, for example, use the information in the “tidx” or “sidx” box, if present, to locate the random access points and the corresponding presentation time of the media presentation. In cases where a segment is a 3GPP movie fragment, it is also possible for the customer to use information within the “moor” and “mdat” boxes, for example to locate RAPs and obtain the necessary time to present the data in the film fragment and the MPD segment start time. If no RAP with submission time before the requested submission time tp is available, the client can either access the previous segment, or can just use the first random access point as the search result. When media segments start with a RAP, these procedures are simple.
[000431]Note also that not necessarily all the media segment information needs to be downloaded to access the presentation time tp. The client can, for example, initially request the “tidx” or “sidx” box from the beginning of the media segment using byte range requests. Using tidx or sidx boxes, segment timing/timing can be mapped to segment byte ranges. By continuous use of partial HTTP requests, only the relevant parts of the media segment need to be accessed, for a better user experience and low launch delays. Segment List Generation
[000432] As described here, it should now be clear how to implement a simple HTTP streaming client, which uses the information provided by the MPD to create a list of segments for a representation that has a signaled segment duration of approximately dur . In some modalities, the client can assign to the media segments within a representation consecutive indexes i = 1, 2, 3,..., that is, the first media segment is assigned the index i = 1, the second segment of media is assigned index i = 2 and so on. Then, the list of media segments with segment indexes i is assigned the startTime[i] and the URL[i] is generated, for example as follows. First, index i is set to 1. The start time of the first media segment is taken as 0, startTime[1] = 0. The URL of media segment i, URL [i], is taken as FileURI (r,i). The process proceeds for all media segments described with index i and the startTime[i] of the media segment is taken as (i-l)*dur and the URL[i] is taken as FileURI(r, i). Simultaneous HTTP/TCP Requests
[000433]A concern in a block request streaming system is a desire to always request the highest quality blocks that can be completely received in time for playback. However, the data arrival rate cannot be known in advance and so it may happen that a requested block does not arrive in time to be played. This results in a need to pause media playback, which results in a poor user experience. This problem can be mitigated by client algorithms that take a conservative approach to the selection of blocks to request, requesting lower quality (and therefore smaller size) blocks, which are more likely to be received on time, even if the arrival rate drops during block reception). However, this conservative approach has the disadvantage of possibly delivering poorer quality playback to the target user or device, which is also tantamount to a poor user experience. The problem can be magnified when multiple HTTP connections are used at the same time to download different blocks, as described below, since available network resources are shared across connections and therefore are being used simultaneously for blocks with different replay times .
[000434] It may be advantageous for the client to issue requests for multiple blocks simultaneously, in which, in this context, "simultaneously" means that the responses to the requests are occurring with overlapping time intervals, not necessarily being the case where the requests are done precisely, or even approximately, at the same time. In the case of the HTTP protocol, this approach can improve the utilization of available bandwidth due to the behavior of the TCP protocol (which is well known). This can be especially important to improve content zapping time, since when new content is first requested the corresponding HTTP/TCP connections through which data is requested for blocks can be slow to initialize, and so use multiple HTTP/TCP connections at this point can dramatically speed up data delivery time of the first blocks. However, requesting different blocks or fragments over different HTTP/TCP connections can also lead to performance degradation as requests for blocks that are to be played for the first time are competing with requests for subsequent blocks, downloads Concurrent HTTP/TCP vary widely in their delivery speed and therefore request completion time can be highly variable, it is generally not possible to control which HTTP/TCP downloads will complete quickly and which will be slower and therefore likely that at least sometimes the HTTP/TCP downloads of the first blocks will be the last to complete, thus leading to large and variable channel zapping times.
[000435]Let's assume that each block or fragment of a segment is downloaded via a separate HTTP/TCP connection and that the number of parallel connections is n and the playback duration of each block is t seconds, with the flow rate continuous content associated with the segment is S. When the client, for the first time, starts playing the content, requests can be issued for the first n blocks, representing n*t seconds of media data.
[000436]As the technicians in the field know, there is a wide variation in the data rate of TCP connections. However, to simplify this discussion, let's assume that ideally all connections are advanced in parallel, such that the first block will be completely received at approximately the same time as the other n-1 requested blocks. To further simplify the discussion, let's assume that the total bandwidth used by the n download connections is fixed and has a value of b for the entire duration of the download, and that the streaming rate s is constant throughout the representation. Let's further assume that the media data structure is such that playback of a block can be done when the entire block is available on the client, that is, playback of a block can only start after the entire block is received, for example due to to the underlying video encoding structure, or because encryption/encoding is being employed to encrypt each fragment or block separately and therefore the entire fragment or block must be received before it can be decoded. So, to simplify the discussion below, we can assume that an entire block must be received before any of the blocks can be played. Therefore, the time required before the first block arrives and can be played is approximately n*t*S/b.
[000437]Since it is desirable to minimize content zapping time, it is therefore desirable to minimize n*t*S/b. The value of t can be determined by factors such as the underlying structure of video encoding and how ingest methods are used; thus t can be reasonably small, but very small values of t lead to an overly complicated segment map and possibly incompatible with efficient video encoding and decoding, if used. The value of n can also affect the value of B, that is, B can be larger for a large number of connections and therefore reducing the number of connections, n, has the potential negative side effect of reducing the amount of width of available bandwidth that is used, B, and thus may not be effective in achieving the goal of reducing content zapping time. The value of s depends on which representation is chosen for download and playback, and ideally s should be as close to b as possible to maximize media/media playback quality for the given network conditions. So, to simplify this discussion, let's assume that s is approximately equal to b. Therefore, the channel zapping time is proportional to n*t. Thus, using more connections to download different fragments can degrade channel zapping time if the total bandwidth used by the connections is sublinearly proportional to the number of connections, which is usually the case.
[000438]As an example, let's assume that t = 1 second and with n = 1 the value of B = 500 Kbps and with n = 2 the value of B = 700 Kbps and with n = 3 the value of B = 800 Kbps. Suppose that the representation with S = 700 Kbps is chosen. So with n = 1 the download time for the first block is 1 * 700/500 = 1.4 seconds, with n = 2 the download time for the first block is 2 * 700/700 = 2 seconds and with n = 3 the download time for the first block is 3 * 700/800 = 2,625 seconds. Also, as the number of connections increases the variability in individual connection download speeds is likely to increase (although even with one connection there is likely some significant variability). Thus, in this example, channel zapping time and the variability in channel zapping time increase as the number of connections increases. Intuitively, the blocks being delivered have different priorities, that is, the first block has the earliest deadline, the second block has the second oldest deadline, and so on. Since the download connections over which blocks are being delivered are competing for network resources during delivery, blocks with earlier deadlines become more delayed as more concurrent blocks are requested. On the other hand, even in this case, ultimately using more than one transfer connection allows to sustainably support higher streaming rate, for example, with three connections a transmission rate of up to 800 Kbps can be supported in this example , whereas only a 500 Kbps stream can be supported with one connection.
[000439]In practice, as noted above, the data rate of a connection can be highly variable both within the same connection over time and between connections and, as a result, the requested n blocks are usually not completed at the same time and , in fact, it might commonly be the case that a block can be completed in half the time of another block. This effect results in unpredictable behavior, as in some cases the first block may complete much earlier than other blocks and in other cases the first block may complete later than other blocks, and thus the start of playback may , in some cases, occur relatively quickly and in other cases may be slow to occur. This unpredictable behavior can be frustrating for the user and therefore can be considered a poor user experience.
[000440] What is needed, therefore, are methods in which multiple TCP connections can be used to improve channel zapping time and channel zapping time variability, while simultaneously supporting a possible good quality streaming rate . What is also needed are methods to allow sharing of the available bandwidth allocated to each block, being adjusted as the playback time of a block approaches, so that, if necessary, a higher percentage of available bandwidth can be allocated to the block with the closest playtime. HTTP/TCP Cooperative Request
[000441] Methods of using simultaneous HTTP/TCP requests in a cooperative manner will now be described. A receiver can employ multiple simultaneous cooperative HTTP/TCP requests, for example using a plurality of HTTP byte range requests, where each such request is for a part of a fragment in a source segment, or all of a fragment of a source segment, or a part of a repair fragment, or for an entire repair fragment of a repair segment.
[000442]The advantages of cooperative HTTP/TCP requests in conjunction with the use of FEC repair data can be especially important in providing consistently fast channel zapping times. For example, at a time of channel zapping it is likely that TCP connections have just started or have been idle for some period of time, in which case the congestion window, cwnd, is at its minimum value for connections, and thus the delivery speed of these TCP connections will take several round-trip times (RTTs) to ramp up, and there will be high variability in the delivery speeds across different TCP connections during this ramp-up time.
[000443] An overview of the “non-FEC” or “no-FEC” method will now be described, which is a cooperative HTTP/TCP request method in which only source block media data is requested using multiple simultaneous HTTP/TCP connections, meaning no FEC repair data is being requested. With the non-FEC method, portions of the same fragment are requested over different connections, for example using HTTP requests from byte ranges to portions of the fragment, for example each HTTP byte range request is a part of the indicated byte range in the segment map for the fragment. There may be the case where an individual HTTP/TCP request accelerates its delivery speed to fully utilize the available bandwidth over several round-trip times and thus there is a relatively long period of time where the speed of delivery will be less than the available bandwidth and thus if a single HTTP/TCP connection is used to download for example the first fragment of a content to be played, the channel zapping time could be large. Using the non-FEC method, downloading different parts of the same fragment over different HTTP/TCP connections can significantly reduce channel zapping time.
[000444] An overview of the FEC method will now be described, which is a cooperative HTTP/TCP request method in which media data from a source segment and FEC repair data generated from the data media is requested using multiple simultaneous HTTP/TCP connections. With the FEC method, portions of the same fragment and FEC repair data generated from that fragment are requested over different connections, using HTTP byte range requests for portions of the fragment, and so, for example, each byte range request HTTP is a part of the byte range indicated in the segment map for the fragment. It can happen that an individual HTTP/TCP request speeds up the delivery speed to fully utilize the available bandwidth over several round-trip times and therefore there is a relatively long period of time where the delivery speed is less than the available bandwidth and thus if a single HTTP/TCP connection is used to download for example the first fragment of a content to be played, the channel zapping time could be large. Using the FEC method has the same advantages as the non-FEC method and has the added advantage that not all the requested data needs to arrive before the fragment can be retrieved, thus further reducing channel zapping time and variability in the channel zapping time. Making requests over different TCP connections, and super-ordering by also requesting FEC repair data through at least one of the connections, the amount of time required to deliver a sufficient amount of data to, for example, retrieve the first requested fragment that allows start media playback to start, can be greatly reduced and made much more consistent than if cooperative TCP connections and FEC repair data were not used.
[000445]Figures 24(a) to (e) present an example of fluctuations in the delivery rate of 5 TCP connections running over the same link to the same client of the same HTTP web server of an EV-DO (Emulated Evolution) network Data Optimized). In Figures 24(a) to (e) the x-axis indicates the time in seconds and the y-axis the rate at which bits are received at the client through each of the 5 TCP connections, measured over 1-second intervals , for each connection. In this particular emulation, there were 12 TCP connections in full gear over this link, so the network was relatively loaded for the time shown, which could be typical when more than one client is streaming within the same cell of a mobile network. Note that although delivery rates are somewhat correlated over time, there is a large difference in delivery rates for the 5 connections at many points in time.
[000446] Figure 25 shows a possible request/request structure for a fragment that is 250,000 bits in size (approximately 31.25 kilobytes), where there are 4 HTTP requests of byte ranges made in parallel for different parts of the fragment, that is, the first HTTP connection requests the first 50,000 bits, the second HTTP connection requests the next 50,000 bits, the third HTTP connection requests the next 50,000 bits, and the fourth HTTP connection requests the next 50,000 bits. If FEC is not used, ie the non-FEC method then these are the only 4 requests for the fragment in this example. If FEC is used, ie the FEC method, then in this example there is an additional HTTP connection that requests an additional 50,000 bits of FEC repair data from a repair segment generated from the fragment.
[000447] Figure 26 is an enlargement of the first pair of seconds of the 5 TCP connections shown in Figures 24(a) to (e) where, in Figure 26 the x-axis shows the time in 100 millisecond intervals, and the The y-axis shows the rate at which bits are received at the client over each of the 5 TCP connections, measured over 100 millisecond intervals. One line shows the total amount of bits that were received by the client for the fragment of the first 4 HTTP connections (excluding the HTTP connection over which FEC data is requested), that is, what is received using the non- FEC. Another line shows the total amount of bits that were received by the client for the fragment for all 5 HTTP connections (including the HTTP connection over which FEC data is requested), ie what arrives using the FEC method. For the FEC method, it is assumed that the fragment can be FEC decoded by receiving any 200,000 bits out of the requested 250,000 bits, which can be done if for example a FEC Reed-Solomon code is used, and that it can be essentially done if, for example, the RaptorQ code described in Luby IV is used. For the FEC method in this example, enough data is received to retrieve the fragment using FEC decoding after 1 second, allowing for a 1 second channel zapping time (assuming that subsequent fragment data can be requested and received before the first fragment is fully reproduced). For the non-FEC method in this example, all data from the 4 requests must be received before the fragment can be retrieved, which occurs after 1.7 seconds, leading to a channel zapping time of 1.7 seconds. Thus, in the example shown in Fig. 26, the non-FEC method is 70% worse in terms of channel zapping time than the FEC method. One of the reasons for the advantage demonstrated by the FEC method in this example is that, for the FEC method, receiving any 80% of the requested data allows for fragment recovery, while for the non-FEC method, receiving 100% of the requested data is required. Thus, the non-FEC method will have to wait for the slower TCP connection to complete the delivery and, because of the natural variations in the TCP delivery rate, there will be wide variation in the delivery speed of the slower TCP connection compared to a connection Average TCP. With the FEC method in this example, a slow TCP connection does not determine when the fragment is recoverable. Instead, for the FEC method, sufficient data delivery is much more a function of the average TCP delivery rate than the worst-case TCP delivery rate.
[000448]There are many variations of the non-FEC method and the FEC method described above. As an example, cooperative HTTP/TCP requests can be used only for the first few fragments after a channel zapping has occurred, and later only a single HTTP/TCP request is used to download more fragments, multiple fragments, or segments whole. As another example, the number of cooperative HTTP/TCP connections used may be a function of how urgently the fragments are being requested, that is, how imminent the timing of playback of these fragments is and current network conditions.
[000449]In some variations, a plurality of HTTP connections can be used to request repair data from repair segments. In other variations, different amounts of data may be requested over different HTTP connections, for example depending on the current media buffer size and the data reception rate on the client. In another variation, the source representations are not independent of each other, but rather represent a layered coding media, where, for example, an enhanced font representation might depend on a base font representation. In this case, it can be a repair representation corresponding to the source base representation and another repair representation corresponding to the combination of base and enhanced source representations.
[000450]Additional global elements add to the advantages that can be obtained by the methods disclosed above. As an example, the number of HTTP connections used may vary depending on the actual amount of media in the media buffer and/or the reception rate for the media buffer. Cooperative HTTP requests using FEC, ie the FEC method described above, and variants of this method can be heavily used when the media buffer is relatively empty. As an example, more cooperative HTTP requests are made in parallel to different parts of the first fragment, requesting the entire source fragment and a relatively large fraction of repair data from the corresponding repair fragment and then transitioning to a reduced number from simultaneous HTTP requests, requesting larger portions of media data per request, and requesting a smaller fraction of repair data, for example, moving to 1, 2, or 3 simultaneous HTTP requests, making requests for full chunks or multiple consecutive chunks per request, and moving to stop requesting repair data as the media buffer grows.
[000451] As another example, the amount of FEC repair data may vary as a function of the media buffer size, ie when the media buffer is small then more FEC repair data may be requested, and the measurement that the media buffer grows, so the amount of FEC repair data requested may decrease and at some point when the media buffer is large enough, then no FEC repair data can be requested, only data from the source segment of the representations of source. The benefits of these advanced techniques are that they can allow for faster and more consistent channel zapping times, as well as more resistance against potential media irregularities and interruptions, while minimizing the amount of additional bandwidth used beyond the amount that would otherwise be. consumed only by provisioning the media on the source segments reducing both request message traffic and FEC repair data, while allowing the support of higher media rates for given network conditions. Additional Enhancements When Using Concurrent HTTP Connections
[000452]An HTTP/TCP request can be abandoned if a suitable condition is found and another HTTP/TCP request can be made to download data that can replace the data requested in the abandoned request, where the second HTTP/TCP request can request exactly the same data as in the original order, eg source data; or an overlay of data, for example, some from the same data source and remediation data that was not requested in the first request; or completely disconnected data, eg repair data that was not requested in the first request. An example of a suitable condition is one where a request fails due to failure to respond from the block server infrastructure (BSI) within a given time period or a failure to establish a transport connection to the BSI or to receive a an explicit server fault message or other fault condition.
[000453] Another example of a suitable condition is that the reception of data is proceeding abnormally slowly, according to a measurement comparison of the connection speed (data arrival rate in response to the request in question) with the expected connection speed or an estimate of the connection speed required to receive the response before the playback time of the media data contained therein, or other time dependent on such time.
[000454] Such an approach has the advantage in the case that the BSI sometimes has failures or poor performance. In this case, the above approach increases the likelihood that the client can continue to reliably play media data despite failures or poor performance within BSI. Note that in some cases there may be advantages in designing the BSI in such a way that it has such flaws or poor performance in certain occasions, for example such a project may have a lower cost than an alternative design that does not have such flaws or poor performance or that presents these less frequently. In this case, the method described here still has an advantage in that it allows the use of a lower cost scheme for BSI without a consequent degradation in the user experience.
[000455]In another modality, the number of requests issued for data corresponding to a given block may be dependent on whether a suitable condition regarding the block is met. If the condition is not met then the client can be restricted to make new requests for the block if the successful completion of all currently incomplete data requests for the block would allow block recovery with high probability. If the condition is met, then a greater number of requests for the block can be issued, ie the above limitation does not apply. An example of a suitable condition is that the time until the block's scheduled playback time, or other time dependent on this time, is below a given threshold. This method has an advantage in that additional data requests for a block are issued when reception of the block becomes more urgent, because the reproduction time of the media data that includes the block is near. In the case of common transport protocols such as HTTP/TCP, these additional requests have the effect of increasing the percentage of available bandwidth dedicated to data contributing to the reception of the block in question. This reduces the time required to receive enough data to recover the block, to complete, and therefore reduces the probability that the block cannot be recovered before the scheduled time for playing the media data that includes the block. As described above, if the block cannot be recovered before the scheduled playback time of the media data that includes the block, playback may pause, resulting in a poor user experience, and therefore the method described here advantageously reduces probability of that bad user experience.
[000456] It should be clear that throughout this descriptive report references to the scheduled time of playback of a block refer to the time when encoded media data comprising the block is first available on the client to obtain playback of the presentation without pausing. As will be clear to technicians in the field of media presentation systems, this occurs in practice just before the actual time of appearance of the media comprising the block in physical transducers used for reproduction (screen, speakers, etc.) given that various transformation functions may need to be applied to the media data that includes the block to effect actual playback of that block, and these functions may require a certain amount of time to complete. As an example, media data is often transported in compressed form and a decompression transformation can be applied. Methods for generating file structures that support cooperative HTTP/FEC methods
[000457] A way to generate a file structure that can be used advantageously by a client employing cooperative HTTP/FEC methods will now be described. In the present embodiment, for each segment source there is a corresponding repair segment generated as follows. The R parameter indicates, on average, how much FEC repair data is generated for the source/source data in the source segments. As an example, R = 0.33 indicates that if a source segment contains 1000 kilobytes of data, the corresponding repair segment contains approximately 330 kilobytes of repair data. The S parameter indicates the symbol size in bytes used for FEC encoding and decoding. As an example, S = 64 indicates that the source data and repair data comprise symbols of 64 bytes each for FEC encoding and decoding.
[000458]Repair segment can be generated for a source segment as follows. Each fragment of the source segment is considered as a source block for FEC encoding purposes, and thus each fragment is treated as a sequence of source symbols from a source block from which repair symbols are generated. The number of repair symbols generated in total for the first i fragments is calculated as TNRS(i) = ceiling(R*B(i)/S), where ceiling(x) is the function that generates the smallest integer with a value that is at least x. Thus, the number of repair symbols generated for fragment i is NRS(i) = TNRS(i) - TNRS (i-1).
[000459] The repair segment comprises a concatenation of the repair symbols to the fragments, in which the order of the repair symbols within a repair segment is in the order of the fragments from which they are generated, and within a fragment the symbols repairs are in the order of their encoding symbol identifier (ESI). The repair segment structure corresponding to a source segment structure is shown in Figure 27, including a repair segment generator 2700.
[000460]Note that by setting the number of repair symbols for a fragment, as described above, the total number of repair symbols for all previous fragments, and therefore the byte index for the repair segment, only it depends on R, S, B(i-1) and B(i), and it does not depend on any of the anterior or posterior structures of the fragments within the segment of origin. This is advantageous because it allows a customer to quickly calculate the start position of a repair block within the repair segment and also quickly calculate the number of repair symbols within that repair block, using only local information about the corresponding fragment structure. from the source segment from which the repair block is generated. Thus, if a client decides to start downloading and replaying a fragment in the middle of a source segment, it can also quickly generate and access the corresponding repair block from within the corresponding repair segment.
[000461]The number of source symbols in the source block corresponding to the fragment is calculated as NSS(i) = ceiling((B(i)-B(i-1))/S). The last source symbol is padded with zero bytes for FEC encoding and decoding purposes if B(i)-B(i-1) is not a multiple of S, ie the last source symbol is padded with zero bytes for it is s bytes in size for FEC encoding and decoding purposes, but these zero padding bytes are not stored as part of the source segment. In such an embodiment, the ESIs for the font symbol are 0, 1,..., NSS(i)-1 and the ESIs for the repair symbols are NSS(i),..., NSS(i)+RN( i)-1.
[000462]The URL for a repair segment in this modality can be generated from the URL for the corresponding source segment simply for example by adding the suffix “.repair” to the source segment URL.
[000463]The repair index information and FEC information for a repair segment are implicitly defined by the index information for the corresponding source segment, and from the values of R and S, as described herein. The time offsets and structure of the fragment that includes the repair segment are determined by the time offsets and structure of the corresponding source segment. The offset/offset of bytes for the end of the repair symbols in the repair segment corresponding to fragment i can be calculated as RB(i) = S*ceiling(R*B(i)/S). The number of bytes in the repair segment corresponding to fragment i is then RB(i)-RB(i-1), and therefore the number of repair symbols corresponding to fragment i is calculated as NRS(i) = (RB (i)-RB(i-1))/S. The number of source symbols corresponding to the fragment can be calculated as NSS(i) = ceiling((B(i)-B(i-1))/S). Thus, in the present embodiment, the repair indexing information for a repair block within a repair segment and the corresponding FEC information can be implicitly derived from R, S and the indexing information for the corresponding segment fragment of corresponding origin.
[000464]As an example, consider the example shown in Figure 28, showing a fragment 2 that starts at byte offset/offset B(1) = 6410 and ends at byte offset/offset B(2) = 6.770. In this example, the symbol size is S = 64 bytes, and the dotted vertical lines show the byte offsets within the source segment that correspond to multiples of S. The total size of the repair segment as a fraction of the segment size of the font is defined as R = 0.5 in this example. The number of source symbols in the source block for fragment 2 is calculated as NSS(2) = ceiling((6.770-6.410)/64) = ceil(5.625) = 6, and such 6 source symbols have ESIs 0, ..., 5, respectively, where the first font symbol consists of the first 64 bytes of fragment 2, which starts at byte index 6410 within the source segment, the second font symbol consists of the next 64 bytes of fragment 2 , which start at byte index 6,474 within the source segment, and so on. The final byte offset of the repair block corresponding to fragment 2 is calculated as RB(2) = 64*ceiling(0.5*6.770/64) = 64*ceiling(52.89...) = 64 * 53 = 3.392, and the starting byte offset of the repair block corresponding to fragment 2 is calculated as RB(1) = 64*ceiling(0.5*6.410/64) = 64*ceiling(50.07...) = 64 * 51 = 3264, and so, in this example, there are two repair symbols in the repair block corresponding to fragment 2 with ESIs 6 and 7, respectively, starting at byte offset 3.264 within the repair segment and ending at byte offset 3.392.
[000465]Note that in the example shown in Figure 28, even though R = 0.5 and there are 6 source symbols corresponding to fragment 2, the number of repair symbols is not 3, as you would expect if you simply used the number of font symbols to calculate the number of repair symbols but 2 according to the methods described here. As opposed to simply using the number of source symbols of a fragment to determine the number of repair symbols, the modalities described above make it possible to calculate the position of the repair block within the repair segment solely from the index information associated with the corresponding source block of the corresponding source segment. Also, as the number, K, of font symbols in a source block grows, the number of repair symbols, KR, of the corresponding repair block is closely approximated by K*R, as in general, KR is at most ceil(K*R) and KR is at least floor((K-1)*R), where floor(x) is the greatest integer that is at most x.
[000466] There are many variations of the above modalities to generate a file structure that can be advantageously used by a client employing cooperative HTTP/FEC methods, as those skilled in the art will note. As an example of an alternative modality, an original segment for a representation can be partitioned into n > 1 parallel segments, where for i = 1,..., N, a specified fraction Fi of the original segment is contained in the ith parallel segment , and where the sum for i = 1,..., N of Fi is equal to 1. In such a modality, there may be a master segment map that is used to derive the segment maps for all parallel segments, so similar to that in which the repair segment map is derived from the source segment map in the modality described above. As an example, the master segment map can indicate the fragment structure if all the data from the source media was not partitioned into parallel segments, but instead contained in an original segment, and the segment map for the 9th parallel segment can be derived from the second master map by calculating that if the amount of media data in a fragment's first prefix of the original segment is L bytes, then the total number of bytes of this prefix in the total between the first i parallel segments is ceil( L*Gi), where Gi is the sum at j = 1,..., i of Fj. As another example of an alternative modality, segments can consist of combining the original source media data for each fragment immediately followed by the repair data for that fragment, resulting in a segment that contains a mixture of repair data generated using a code FEC from the source media data. As another example of an alternative embodiment, a segment containing a mixture of source and repair media data can be partitioned into multiple parallel segments containing a mixture of repair data and source media data.
[000467]Other modalities can be imagined by technicians in the area after reading this disclosure. In other embodiments, combinations or sub-combinations of the invention disclosed above can be advantageously made. Exemplary arrangements of components are presented for purposes of illustration and it should be clear what combinations, additions, rearrangements, and so forth are contemplated in alternative embodiments of the present invention. Thus, although the invention has been described in exemplary embodiments, those skilled in the art will recognize that various modifications are possible.
[000468]As an example, the processes described here can be implemented using hardware components, software components and/or any combination of such. In some cases, software components may be provided through tangible, non-transient media for execution on hardware that is provided with the media, or separate from the media. The descriptive report and the drawings must, therefore, be considered as illustrative and not restrictive. However, it will be evident that various modifications and changes may be made thereto without constituting a departure from the broader spirit and scope of the invention as defined in the claims and that the invention is intended to cover all modifications and equivalents falling within the scope of the claims that follow.

权利要求:
Claims (15)
[0001]
1. Method for generating blocks of media data to be transmitted electronically to client devices upon request, the method characterized by comprising: - obtaining (103) data representing media from a presentation, where the presentation consists of a presentation throughout the tempo and has a defined playback rate and parts of the performance can be defined by time ranges; - storing (110) data representing presentation media as a plurality of segments (510), each segment (510) comprising a plurality of blocks (F, Blk) and stored matching data; - identifying, before transmitting a subset of the plurality of blocks, correspondences between a plurality of time bands and a plurality of positions within blocks within the segments; - generating, prior to transmission of the subset of the plurality of blocks, the stored match data (S) representing at least some of the matches, such that a client device can determine from the stored match data and a desired time range of the presentation to be reproduced, which subset to request of the plurality of blocks.
[0002]
Method according to claim 1, characterized in that the stored correspondence data is stored as part of a file which also contains the media data.
[0003]
3. Method according to claim 1, characterized in that the stored correspondence data constitute a map formatted as XML metadata, in which the time range is relative to the beginning of the presentation or relative to the beginning of a block of media.
[0004]
4. Method according to claim 1, characterized by generating blocks of media data and the stored correspondence data are performed by a media ingestion system and stored on a general purpose server that responds at least to file requests .
[0005]
5. Method according to claim 4, characterized in that the file requests are HTTP requests.
[0006]
6. Method according to claim 1, characterized in that the media blocks have variable duration and the stored match data allows client devices to determine data location matches and time ranges that may vary depending on the varying durations of media blocks .
[0007]
7. Method according to claim 1, characterized in that a group of images, GOP, is partitioned into more than one block of media.
[0008]
8. Method for determining requests to perform, on a client device that is capable of presenting a media presentation during a presentation time period, the method characterized by comprising: - determining, on the client device, a desired time period of the media presentation, where the desired time period is less than the total presentation time period; - obtain, on the client device, stored correspondence data (S) that maps time ranges of the media presentation to data ranges within segments representing the media, wherein each segment (510) comprises a plurality of blocks (F, Blk) and stored correspondence data (S); - determining (123), in the client device, and from the stored correspondence data and the desired period of time, which data ranges of the blocks to request from the media server; - carry out (124) the specified requests; and - present (128) the desired time period of the presentation using the client device.
[0009]
Method according to claim 8, characterized in that the stored correspondence data is stored as part of a file which also contains the media data.
[0010]
10. Method according to claim 8, characterized in that the stored correspondence data constitute a map formatted as XML metadata, in which the time range is relative to the beginning of the presentation or relative to the beginning of the media block.
[0011]
11. Method according to claim 8, characterized in that blocks of media data and the stored correspondence data are stored on a general purpose server that responds at least to file requests.
[0012]
12. Method according to claim 11, characterized in that the file requests are HTTP requests.
[0013]
13. Apparatus for generating blocks of media data to be transmitted electronically to client devices upon request, the apparatus characterized by comprising: - mechanisms for obtaining (103) data representing media from a presentation, wherein the presentation consists of a presentation to the over time and has a set playback rate and parts of the performance can be set by time ranges; - mechanisms for storing (110) the data representing presentation media as a plurality of segments (510), each segment (510) comprising a plurality of blocks (F, Blk) and stored correspondence data (S); - mechanisms for identifying, before transmission of a subset of blocks, correspondences between a plurality of time bands and a plurality of positions of the blocks within the segments; - mechanisms for generating, prior to transmission of the subsets of blocks, stored match data (S) representing at least some of the matches, such that a client device can determine from the stored match data and a desired time range of the presentation to be reproduced, which subset to request of the plurality of blocks within the segments.
[0014]
14. Client device capable of presenting a media presentation during a presentation time period, to determine requests to be carried out, the client device characterized by comprising: - mechanisms for determining, on the client device, a desired time period of the presentation of media, where the desired time period is less than the total presentation time period; - mechanisms to obtain, on the client device, stored correspondence data (S) that map time ranges of the media presentation to data ranges of a plurality of media data blocks within segments representing the media, where each segment ( 510) comprises a plurality of blocks (F, Blk) and the stored correspondence data; - mechanisms for determining (123), in the client device, and from the stored correspondence data and the desired period of time, which data tracks of the plurality of blocks within the segments to request; - mechanisms to carry out (124) the determined requests; and - mechanisms for presenting (128) the desired time period of the presentation using the client device.
[0015]
15. Memory characterized by comprising executable instructions which, when executed, cause at least one computer to execute a method as defined in any one of claims 1 to 12.

类似技术:

---

公开号 | 公开日 | 专利标题

---

JP2019205175A|2019-11-28|Enhanced block-request streaming system using signaling or block creation

---

JP2017118553A|2017-06-29|Enhanced block-request streaming using cooperative parallel http and forward error correction

---

KR101741484B1|2017-05-30|Enhanced block-request streaming system for handling low-latency streaming

---

EP2481195B1|2019-10-30|Enhanced block-request streaming using url templates and construction rules

---

JP5722331B2|2015-05-20|Extended block-request streaming with scalable coding

---

JP5823396B2|2015-11-25|Enhanced block-request streaming with block partitioning or request control for improved client-side processing

---

BR112014026741B1|2021-10-26|METHOD FOR STRUCTURING THE CONTENT DATA TO BE SERVED USING A MEDIA SERVER, MEDIA SERVER AND COMPUTER-READABLE MEMORY

同族专利:

---

公开号 | 公开日

---

RU2012116086A|2013-10-27|

---

EP2481197A2|2012-08-01|

---

JP5666598B2|2015-02-12|

---

KR20120080208A|2012-07-16|

---

JP2016174363A|2016-09-29|

---

PT2481197T|2019-07-09|

---

CA2854017C|2017-11-21|

---

ZA201202935B|2012-12-27|

---

CA2774923A1|2011-03-31|

---

JP2021073774A|2021-05-13|

---

EP2481197B1|2019-04-10|

---

JP2018088679A|2018-06-07|

---

US20160323342A1|2016-11-03|

---

WO2011038013A3|2011-08-11|

---

WO2011038013A2|2011-03-31|

---

SI2481197T1|2019-05-31|

---

JP2019205175A|2019-11-28|

---

CA2854017A1|2011-03-31|

---

CA2854011A1|2011-03-31|

---

JP5911926B2|2016-04-27|

---

DK2481197T3|2019-07-22|

---

BR112012006374A2|2017-07-18|

---

US20110238789A1|2011-09-29|

---

KR101395193B1|2014-05-15|

---

KR101562274B1|2015-10-21|

---

KR101456987B1|2014-11-04|

---

KR20140069369A|2014-06-09|

---

JP6685989B2|2020-04-22|

---

US9432433B2|2016-08-30|

---

JP2013505680A|2013-02-14|

---

CN102577411B|2016-06-29|

---

CA2854011C|2016-11-29|

---

CN102577411A|2012-07-11|

---

MY163822A|2017-10-31|

---

KR101534576B1|2015-07-09|

---

JP2015053677A|2015-03-19|

---

ES2734257T3|2019-12-05|

---

CA2854008C|2017-06-27|

---

CA2774923C|2016-04-19|

---

JP6852121B2|2021-03-31|

---

KR20140004262A|2014-01-10|

---

CA2854008A1|2011-03-31|

---

HUE044122T2|2019-09-30|

---

PL2481197T3|2019-09-30|

---

KR20140069368A|2014-06-09|

---

RU2553101C2|2015-06-10|

---

JP6254208B2|2017-12-27|

引用文献:

---

公开号 | 申请日 | 公开日 | 申请人 | 专利标题

---

---

US3909721A|1972-01-31|1975-09-30|Signatron|Signal processing system|

---

US4365338A|1980-06-27|1982-12-21|Harris Corporation|Technique for high rate digital transmission over a dynamic dispersive channel|

---

US4965825A|1981-11-03|1990-10-23|The Personalized Mass Media Corporation|Signal processing apparatus and methods|

---

US4589112A|1984-01-26|1986-05-13|International Business Machines Corporation|System for multiple error detection with single and double bit error correction|

---

US4901319A|1988-03-18|1990-02-13|General Electric Company|Transmission system with adaptive interleaving|

---

GB8815978D0|1988-07-05|1988-08-10|British Telecomm|Method & apparatus for encoding decoding & transmitting data in compressed form|

---

US5136592A|1989-06-28|1992-08-04|Digital Equipment Corporation|Error detection and correction system for long burst errors|

---

US5421031A|1989-08-23|1995-05-30|Delta Beta Pty. Ltd.|Program transmission optimisation|

---

US7594250B2|1992-04-02|2009-09-22|Debey Henry C|Method and system of program transmission optimization using a redundant transmission sequence|

---

US5701582A|1989-08-23|1997-12-23|Delta Beta Pty. Ltd.|Method and apparatus for efficient transmissions of programs|

---

US5329369A|1990-06-01|1994-07-12|Thomson Consumer Electronics, Inc.|Asymmetric picture compression|

---

US5455823A|1990-11-06|1995-10-03|Radio Satellite Corporation|Integrated communications terminal|

---

US5164963A|1990-11-07|1992-11-17|At&T Bell Laboratories|Coding for digital transmission|

---

US5465318A|1991-03-28|1995-11-07|Kurzweil Applied Intelligence, Inc.|Method for generating a speech recognition model for a non-vocabulary utterance|

---

EP0543070A1|1991-11-21|1993-05-26|International Business Machines Corporation|Coding system and method using quaternary codes|

---

US5379297A|1992-04-09|1995-01-03|Network Equipment Technologies, Inc.|Concurrent multi-channel segmentation and reassembly processors for asynchronous transfer mode|

---

US5371532A|1992-05-15|1994-12-06|Bell Communications Research, Inc.|Communications architecture and method for distributing information services|

---

US5425050A|1992-10-23|1995-06-13|Massachusetts Institute Of Technology|Television transmission system using spread spectrum and orthogonal frequency-division multiplex|

---

US5372532A|1993-01-26|1994-12-13|Robertson, Jr.; George W.|Swivel head cap connector|

---

EP0613249A1|1993-02-12|1994-08-31|Altera Corporation|Custom look-up table with reduced number of architecture bits|

---

DE4316297C1|1993-05-14|1994-04-07|Fraunhofer Ges Forschung|Audio signal frequency analysis method - using window functions to provide sample signal blocks subjected to Fourier analysis to obtain respective coefficients.|

---

AU665716B2|1993-07-05|1996-01-11|Mitsubishi Denki Kabushiki Kaisha|A transmitter for encoding error correction codes and a receiver for decoding error correction codes on a transmission frame|

---

US5590405A|1993-10-29|1996-12-31|Lucent Technologies Inc.|Communication technique employing variable information transmission|

---

JP2576776B2|1993-11-10|1997-01-29|日本電気株式会社|Packet transmission method and packet transmission device|

---

US5517508A|1994-01-26|1996-05-14|Sony Corporation|Method and apparatus for detection and error correction of packetized digital data|

---

CA2140850C|1994-02-24|1999-09-21|Howard Paul Katseff|Networked system for display of multimedia presentations|

---

US5566208A|1994-03-17|1996-10-15|Philips Electronics North America Corp.|Encoder buffer having an effective size which varies automatically with the channel bit-rate|

---

US5432787A|1994-03-24|1995-07-11|Loral Aerospace Corporation|Packet data transmission system with adaptive data recovery method|

---

US5757415A|1994-05-26|1998-05-26|Sony Corporation|On-demand data transmission by dividing input data into blocks and each block into sub-blocks such that the sub-blocks are re-arranged for storage to data storage means|

---

US5802394A|1994-06-06|1998-09-01|Starlight Networks, Inc.|Method for accessing one or more streams in a video storage system using multiple queues and maintaining continuity thereof|

---

US5568614A|1994-07-29|1996-10-22|International Business Machines Corporation|Data streaming between peer subsystems of a computer system|

---

US5739864A|1994-08-24|1998-04-14|Macrovision Corporation|Apparatus for inserting blanked formatted fingerprint data in to a video signal|

---

US5668948A|1994-09-08|1997-09-16|International Business Machines Corporation|Media streamer with control node enabling same isochronous streams to appear simultaneously at output ports or different streams to appear simultaneously at output ports|

---

US5926205A|1994-10-19|1999-07-20|Imedia Corporation|Method and apparatus for encoding and formatting data representing a video program to provide multiple overlapping presentations of the video program|

---

US5659614A|1994-11-28|1997-08-19|Bailey, Iii; John E.|Method and system for creating and storing a backup copy of file data stored on a computer|

---

US5617541A|1994-12-21|1997-04-01|International Computer Science Institute|System for packetizing data encoded corresponding to priority levels where reconstructed data corresponds to fractionalized priority level and received fractionalized packets|

---

JP3614907B2|1994-12-28|2005-01-26|株式会社東芝|Data retransmission control method and data retransmission control system|

---

CA2219379A1|1995-04-27|1996-10-31|Cadathur V. Chakravarthy|High integrity transport for time critical multimedia networking applications|

---

US5835165A|1995-06-07|1998-11-10|Lsi Logic Corporation|Reduction of false locking code words in concatenated decoders|

---

US5805825A|1995-07-26|1998-09-08|Intel Corporation|Method for semi-reliable, unidirectional broadcast information services|

---

US6079041A|1995-08-04|2000-06-20|Sanyo Electric Co., Ltd.|Digital modulation circuit and digital demodulation circuit|

---

US5754563A|1995-09-11|1998-05-19|Ecc Technologies, Inc.|Byte-parallel system for implementing reed-solomon error-correcting codes|

---

KR0170298B1|1995-10-10|1999-04-15|김광호|A recording method of digital video tape|

---

US5751336A|1995-10-12|1998-05-12|International Business Machines Corporation|Permutation based pyramid block transmission scheme for broadcasting in video-on-demand storage systems|

---

JP3305183B2|1996-01-12|2002-07-22|株式会社東芝|Digital broadcast receiving terminal|

---

US6012159A|1996-01-17|2000-01-04|Kencast, Inc.|Method and system for error-free data transfer|

---

US5852565A|1996-01-30|1998-12-22|Demografx|Temporal and resolution layering in advanced television|

---

US5936659A|1996-01-31|1999-08-10|Telcordia Technologies, Inc.|Method for video delivery using pyramid broadcasting|

---

US5903775A|1996-06-06|1999-05-11|International Business Machines Corporation|Method for the sequential transmission of compressed video information at varying data rates|

---

US5745504A|1996-06-25|1998-04-28|Telefonaktiebolaget Lm Ericsson|Bit error resilient variable length code|

---

US5940863A|1996-07-26|1999-08-17|Zenith Electronics Corporation|Apparatus for de-rotating and de-interleaving data including plural memory devices and plural modulo memory address generators|

---

US5936949A|1996-09-05|1999-08-10|Netro Corporation|Wireless ATM metropolitan area network|

---

EP0854650A3|1997-01-17|2001-05-02|NOKIA TECHNOLOGY GmbH|Method for addressing a service in digital video broadcasting|

---

KR100261706B1|1996-12-17|2000-07-15|가나이 쓰도무|Digital broadcasting signal receiving device and, receiving and recording/reproducing apparatus|

---

US6044485A|1997-01-03|2000-03-28|Ericsson Inc.|Transmitter method and transmission system using adaptive coding based on channel characteristics|

---

US6011590A|1997-01-03|2000-01-04|Ncr Corporation|Method of transmitting compressed information to minimize buffer space|

---

US6141053A|1997-01-03|2000-10-31|Saukkonen; Jukka I.|Method of optimizing bandwidth for transmitting compressed video data streams|

---

US5946357A|1997-01-17|1999-08-31|Telefonaktiebolaget L M Ericsson|Apparatus, and associated method, for transmitting and receiving a multi-stage, encoded and interleaved digital communication signal|

---

US5983383A|1997-01-17|1999-11-09|Qualcom Incorporated|Method and apparatus for transmitting and receiving concatenated code data|

---

US6014706A|1997-01-30|2000-01-11|Microsoft Corporation|Methods and apparatus for implementing control functions in a streamed video display system|

---

EP1024672A1|1997-03-07|2000-08-02|Sanyo Electric Co., Ltd.|Digital broadcast receiver and display|

---

US6115420A|1997-03-14|2000-09-05|Microsoft Corporation|Digital video signal encoder and encoding method|

---

DE19716011A1|1997-04-17|1998-10-22|Abb Research Ltd|Method and device for transmitting information via power supply lines|

---

US6226259B1|1997-04-29|2001-05-01|Canon Kabushiki Kaisha|Device and method for transmitting information device and method for processing information|

---

US5970098A|1997-05-02|1999-10-19|Globespan Technologies, Inc.|Multilevel encoder|

---

US5844636A|1997-05-13|1998-12-01|Hughes Electronics Corporation|Method and apparatus for receiving and recording digital packet data|

---

JPH1141211A|1997-05-19|1999-02-12|Sanyo Electric Co Ltd|Digital modulatin circuit and its method, and digital demodulation circuit and its method|

---

EP0933768A4|1997-05-19|2000-10-04|Sanyo Electric Co|Digital modulation and digital demodulation|

---

JP4110593B2|1997-05-19|2008-07-02|ソニー株式会社|Signal recording method and signal recording apparatus|

---

US6128649A|1997-06-02|2000-10-03|Nortel Networks Limited|Dynamic selection of media streams for display|

---

US6081907A|1997-06-09|2000-06-27|Microsoft Corporation|Data delivery system and method for delivering data and redundant information over a unidirectional network|

---

US5917852A|1997-06-11|1999-06-29|L-3 Communications Corporation|Data scrambling system and method and communications system incorporating same|

---

KR100240869B1|1997-06-25|2000-01-15|윤종용|Data transmission method for dual diversity system|

---

US5933056A|1997-07-15|1999-08-03|Exar Corporation|Single pole current mode common-mode feedback circuit|

---

US6175944B1|1997-07-15|2001-01-16|Lucent Technologies Inc.|Methods and apparatus for packetizing data for transmission through an erasure broadcast channel|

---

US6047069A|1997-07-17|2000-04-04|Hewlett-Packard Company|Method and apparatus for preserving error correction capabilities during data encryption/decryption|

---

US6904110B2|1997-07-31|2005-06-07|Francois Trans|Channel equalization system and method|

---

US6178536B1|1997-08-14|2001-01-23|International Business Machines Corporation|Coding scheme for file backup and systems based thereon|

---

FR2767940A1|1997-08-29|1999-03-05|Canon Kk|CODING AND DECODING METHODS AND DEVICES AND APPARATUSES IMPLEMENTING THE SAME|

---

EP0903955A1|1997-09-04|1999-03-24|STMicroelectronics S.r.l.|Modular architecture PET decoder for ATM networks|

---

US6088330A|1997-09-09|2000-07-11|Bruck; Joshua|Reliable array of distributed computing nodes|

---

US6134596A|1997-09-18|2000-10-17|Microsoft Corporation|Continuous media file server system and method for scheduling network resources to play multiple files having different data transmission rates|

---

US6272658B1|1997-10-27|2001-08-07|Kencast, Inc.|Method and system for reliable broadcasting of data files and streams|

---

US6081909A|1997-11-06|2000-06-27|Digital Equipment Corporation|Irregularly graphed encoding technique|

---

US6195777B1|1997-11-06|2001-02-27|Compaq Computer Corporation|Loss resilient code with double heavy tailed series of redundant layers|

---

US6073250A|1997-11-06|2000-06-06|Luby; Michael G.|Loss resilient decoding technique|

---

US6163870A|1997-11-06|2000-12-19|Compaq Computer Corporation|Message encoding with irregular graphing|

---

US6081918A|1997-11-06|2000-06-27|Spielman; Daniel A.|Loss resilient code with cascading series of redundant layers|

---

JP3472115B2|1997-11-25|2003-12-02|Ｋｄｄｉ株式会社|Video data transmission method and apparatus using multi-channel|

---

US6243846B1|1997-12-12|2001-06-05|3Com Corporation|Forward error correction system for packet based data and real time media, using cross-wise parity calculation|

---

US5870412A|1997-12-12|1999-02-09|3Com Corporation|Forward error correction system for packet based real time media|

---

US6849803B1|1998-01-15|2005-02-01|Arlington Industries, Inc.|Electrical connector|

---

US6097320A|1998-01-20|2000-08-01|Silicon Systems, Inc.|Encoder/decoder system with suppressed error propagation|

---

US6226301B1|1998-02-19|2001-05-01|Nokia Mobile Phones Ltd|Method and apparatus for segmentation and assembly of data frames for retransmission in a telecommunications system|

---

US6141788A|1998-03-13|2000-10-31|Lucent Technologies Inc.|Method and apparatus for forward error correction in packet networks|

---

US6278716B1|1998-03-23|2001-08-21|University Of Massachusetts|Multicast with proactive forward error correction|

---

JP2002510947A|1998-04-02|2002-04-09|サーノフコーポレイション|Burst data transmission of compressed video data|

---

US6185265B1|1998-04-07|2001-02-06|Worldspace Management Corp.|System for time division multiplexing broadcast channels with R-1/2 or R-3/4 convolutional coding for satellite transmission via on-board baseband processing payload or transparent payload|

---

US6067646A|1998-04-17|2000-05-23|Ameritech Corporation|Method and system for adaptive interleaving|

---

US6018359A|1998-04-24|2000-01-25|Massachusetts Institute Of Technology|System and method for multicast video-on-demand delivery system|

---

US6445717B1|1998-05-01|2002-09-03|Niwot Networks, Inc.|System for recovering lost information in a data stream|

---

US6421387B1|1998-05-15|2002-07-16|North Carolina State University|Methods and systems for forward error correction based loss recovery for interactive video transmission|

---

US6937618B1|1998-05-20|2005-08-30|Sony Corporation|Separating device and method and signal receiving device and method|

---

US6333926B1|1998-08-11|2001-12-25|Nortel Networks Limited|Multiple user CDMA basestation modem|

---

EP1110344A1|1998-09-04|2001-06-27|AT&T Corp.|Combined channel coding and space-block coding in a multi-antenna arrangement|

---

US6415326B1|1998-09-15|2002-07-02|Microsoft Corporation|Timeline correlation between multiple timeline-altered media streams|

---

US7243285B2|1998-09-23|2007-07-10|Digital Fountain, Inc.|Systems and methods for broadcasting information additive codes|

---

US6307487B1|1998-09-23|2001-10-23|Digital Fountain, Inc.|Information additive code generator and decoder for communication systems|

---

US6320520B1|1998-09-23|2001-11-20|Digital Fountain|Information additive group code generator and decoder for communications systems|

---

US6704370B1|1998-10-09|2004-03-09|Nortel Networks Limited|Interleaving methodology and apparatus for CDMA|

---

IT1303735B1|1998-11-11|2001-02-23|Falorni Italia Farmaceutici S|CROSS-LINKED HYALURONIC ACIDS AND THEIR MEDICAL USES.|

---

US6408128B1|1998-11-12|2002-06-18|Max Abecassis|Replaying with supplementary information a segment of a video|

---

US6483736B2|1998-11-16|2002-11-19|Matrix Semiconductor, Inc.|Vertically stacked field programmable nonvolatile memory and method of fabrication|

---

JP2000151426A|1998-11-17|2000-05-30|Toshiba Corp|Interleave and de-interleave circuit|

---

US6166544A|1998-11-25|2000-12-26|General Electric Company|MR imaging system with interactive image contrast control|

---

US6876623B1|1998-12-02|2005-04-05|Agere Systems Inc.|Tuning scheme for code division multiplex broadcasting system|

---

JP3464981B2|1998-12-03|2003-11-10|フラウンホーファー−ゲゼルシャフト・ツール・フェルデルング・デル・アンゲヴァンテン・フォルシュング・アインゲトラーゲネル・フェライン|Information transmitting apparatus and method, and information receiving apparatus and method|

---

US6637031B1|1998-12-04|2003-10-21|Microsoft Corporation|Multimedia presentation latency minimization|

---

US6496980B1|1998-12-07|2002-12-17|Intel Corporation|Method of providing replay on demand for streaming digital multimedia|

---

US6223324B1|1999-01-05|2001-04-24|Agere Systems Guardian Corp.|Multiple program unequal error protection for digital audio broadcasting and other applications|

---

JP3926499B2|1999-01-22|2007-06-06|株式会社日立国際電気|Convolutional code soft decision decoding receiver|

---

US6618451B1|1999-02-13|2003-09-09|Altocom Inc|Efficient reduced state maximum likelihood sequence estimator|

---

US6041001A|1999-02-25|2000-03-21|Lexar Media, Inc.|Method of increasing data reliability of a flash memory device without compromising compatibility|

---

WO2000052600A1|1999-03-03|2000-09-08|Sony Corporation|Transmitter, receiver, transmitter/receiver system, transmission method and reception method|

---

US6466698B1|1999-03-25|2002-10-15|The United States Of America As Represented By The Secretary Of The Navy|Efficient embedded image and video compression system using lifted wavelets|

---

JP3256517B2|1999-04-06|2002-02-12|インターナショナル・ビジネス・マシーンズ・コーポレーション|Encoding circuit, circuit, parity generation method, and storage medium|

---

US6609223B1|1999-04-06|2003-08-19|Kencast, Inc.|Method for packet-level fec encoding, in which on a source packet-by-source packet basis, the error correction contributions of a source packet to a plurality of wildcard packets are computed, and the source packet is transmitted thereafter|

---

US6535920B1|1999-04-06|2003-03-18|Microsoft Corporation|Analyzing, indexing and seeking of streaming information|

---

US6804202B1|1999-04-08|2004-10-12|Lg Information And Communications, Ltd.|Radio protocol for mobile communication system and method|

---

US7885340B2|1999-04-27|2011-02-08|Realnetworks, Inc.|System and method for generating multiple synchronized encoded representations of media data|

---

FI113124B|1999-04-29|2004-02-27|Nokia Corp|Communication|

---

MY130203A|1999-05-06|2007-06-29|Sony Corp|Methods and apparatus for data processing, methods and apparatus for data reproducing and recording media|

---

KR100416996B1|1999-05-10|2004-02-05|삼성전자주식회사|Variable-length data transmitting and receiving apparatus in accordance with radio link protocol for a mobile telecommunication system and method thereof|

---

US6154452A|1999-05-26|2000-11-28|Xm Satellite Radio Inc.|Method and apparatus for continuous cross-channel interleaving|

---

AU5140200A|1999-05-26|2000-12-18|Enounce, Incorporated|Method and apparatus for controlling time-scale modification during multi-media broadcasts|

---

US6229824B1|1999-05-26|2001-05-08|Xm Satellite Radio Inc.|Method and apparatus for concatenated convolutional endcoding and interleaving|

---

JP2000353969A|1999-06-11|2000-12-19|Sony Corp|Receiver for digital voice broadcasting|

---

US6577599B1|1999-06-30|2003-06-10|Sun Microsystems, Inc.|Small-scale reliable multicasting|

---

US20050160272A1|1999-10-28|2005-07-21|Timecertain, Llc|System and method for providing trusted time in content of digital data files|

---

IL141800D0|1999-07-06|2002-03-10|Samsung Electronics Co Ltd|Rate matching device and method for a data communication system|

---

US6643332B1|1999-07-09|2003-11-04|Lsi Logic Corporation|Method and apparatus for multi-level coding of digital signals|

---

US6279072B1|1999-07-22|2001-08-21|Micron Technology, Inc.|Reconfigurable memory with selectable error correction storage|

---

JP3451221B2|1999-07-22|2003-09-29|日本無線株式会社|Error correction coding apparatus, method and medium, and error correction code decoding apparatus, method and medium|

---

US6453440B1|1999-08-04|2002-09-17|Sun Microsystems, Inc.|System and method for detecting double-bit errors and for correcting errors due to component failures|

---

JP2001060934A|1999-08-20|2001-03-06|Matsushita Electric Ind Co Ltd|Ofdm communication equipment|

---

US6430233B1|1999-08-30|2002-08-06|Hughes Electronics Corporation|Single-LNB satellite data receiver|

---

US6332163B1|1999-09-01|2001-12-18|Accenture, Llp|Method for providing communication services over a computer network system|

---

JP4284774B2|1999-09-07|2009-06-24|ソニー株式会社|Transmission device, reception device, communication system, transmission method, and communication method|

---

JP2001094625A|1999-09-27|2001-04-06|Canon Inc|Data communication unit, data communication method and storage medium|

---

WO2001024474A1|1999-09-27|2001-04-05|Koninklijke Philips Electronics N.V.|Partitioning of file for emulating streaming|

---

US6850252B1|1999-10-05|2005-02-01|Steven M. Hoffberg|Intelligent electronic appliance system and method|

---

US7529806B1|1999-11-04|2009-05-05|Koninklijke Philips Electronics N.V.|Partitioning of MP3 content file for emulating streaming|

---

US6523147B1|1999-11-11|2003-02-18|Ibiquity Digital Corporation|Method and apparatus for forward error correction coding for an AM in-band on-channel digital audio broadcasting system|

---

US6785323B1|1999-11-22|2004-08-31|Ipr Licensing, Inc.|Variable rate coding for forward link|

---

US6748441B1|1999-12-02|2004-06-08|Microsoft Corporation|Data carousel receiving and caching|

---

US6678855B1|1999-12-02|2004-01-13|Microsoft Corporation|Selecting K in a data transmission carousel using forward error correction|

---

US6798791B1|1999-12-16|2004-09-28|Agere Systems Inc|Cluster frame synchronization scheme for a satellite digital audio radio system|

---

US6487692B1|1999-12-21|2002-11-26|Lsi Logic Corporation|Reed-Solomon decoder|

---

US20020009137A1|2000-02-01|2002-01-24|Nelson John E.|Three-dimensional video broadcasting system|

---

US6965636B1|2000-02-01|2005-11-15|2Wire, Inc.|System and method for block error correction in packet-based digital communications|

---

US7304990B2|2000-02-03|2007-12-04|Bandwiz Inc.|Method of encoding and transmitting data over a communication medium through division and segmentation|

---

IL140504D0|2000-02-03|2002-02-10|Bandwiz Inc|Broadcast system|

---

WO2001057667A1|2000-02-03|2001-08-09|Bandwiz, Inc.|Data streaming|

---

JP2001251287A|2000-02-24|2001-09-14|Geneticware Corp Ltd|Confidential transmitting method using hardware protection inside secret key and variable pass code|

---

US6765866B1|2000-02-29|2004-07-20|Mosaid Technologies, Inc.|Link aggregation|

---

DE10009443A1|2000-02-29|2001-08-30|Philips Corp Intellectual Pty|Receiver and method for detecting and decoding a DQPSK-modulated and channel-coded received signal|

---

US6384750B1|2000-03-23|2002-05-07|Mosaid Technologies, Inc.|Multi-stage lookup for translating between signals of different bit lengths|

---

JP2001274776A|2000-03-24|2001-10-05|Toshiba Corp|Information data transmission system and its transmitter and receiver|

---

US6510177B1|2000-03-24|2003-01-21|Microsoft Corporation|System and method for layered video coding enhancement|

---

US6851086B2|2000-03-31|2005-02-01|Ted Szymanski|Transmitter, receiver, and coding scheme to increase data rate and decrease bit error rate of an optical data link|

---

US6473010B1|2000-04-04|2002-10-29|Marvell International, Ltd.|Method and apparatus for determining error correction code failure rate for iterative decoding algorithms|

---

US8572646B2|2000-04-07|2013-10-29|Visible World Inc.|System and method for simultaneous broadcast for personalized messages|

---

EP1273152B1|2000-04-08|2006-08-02|Sun Microsystems, Inc.|Method of streaming a single media track to multiple clients|

---

US6631172B1|2000-05-01|2003-10-07|Lucent Technologies Inc.|Efficient list decoding of Reed-Solomon codes for message recovery in the presence of high noise levels|

---

US6742154B1|2000-05-25|2004-05-25|Ciena Corporation|Forward error correction codes for digital optical network optimization|

---

US6694476B1|2000-06-02|2004-02-17|Vitesse Semiconductor Corporation|Reed-solomon encoder and decoder|

---

US6738942B1|2000-06-02|2004-05-18|Vitesse Semiconductor Corporation|Product code based forward error correction system|

---

US7373413B1|2000-06-28|2008-05-13|Cisco Technology, Inc.|Devices and methods for minimizing start up delay in transmission of streaming media|

---

GB2366159B|2000-08-10|2003-10-08|Mitel Corp|Combination reed-solomon and turbo coding|

---

US6834342B2|2000-08-16|2004-12-21|Eecad, Inc.|Method and system for secure communication over unstable public connections|

---

KR100447162B1|2000-08-19|2004-09-04|엘지전자 주식회사|Method for length indicator inserting in protocol data unit of radio link control|

---

JP2002073625A|2000-08-24|2002-03-12|Nippon Hoso Kyokai <Nhk>|Method server and medium for providing information synchronously with broadcast program|

---

US7340664B2|2000-09-20|2008-03-04|Lsi Logic Corporation|Single engine turbo decoder with single frame size buffer for interleaving/deinterleaving|

---

US7031257B1|2000-09-22|2006-04-18|Lucent Technologies Inc.|Radio link protocol /point-to-point protocol design that passes corrupted data and error location information among layers in a wireless data transmission protocol|

---

US7151754B1|2000-09-22|2006-12-19|Lucent Technologies Inc.|Complete user datagram protocol for wireless multimedia packet networks using improved packet level forward error correction coding|

---

US6486803B1|2000-09-22|2002-11-26|Digital Fountain, Inc.|On demand encoding with a window|

---

US7490344B2|2000-09-29|2009-02-10|Visible World, Inc.|System and method for seamless switching|

---

US6763392B1|2000-09-29|2004-07-13|Microsoft Corporation|Media streaming methods and arrangements|

---

US6411223B1|2000-10-18|2002-06-25|Digital Fountain, Inc.|Generating high weight encoding symbols using a basis|

---

US7613183B1|2000-10-31|2009-11-03|Foundry Networks, Inc.|System and method for router data aggregation and delivery|

---

US6694478B1|2000-11-07|2004-02-17|Agere Systems Inc.|Low delay channel codes for correcting bursts of lost packets|

---

US6732325B1|2000-11-08|2004-05-04|Digeo, Inc.|Error-correction with limited working storage|

---

US20020133247A1|2000-11-11|2002-09-19|Smith Robert D.|System and method for seamlessly switching between media streams|

---

US7072971B2|2000-11-13|2006-07-04|Digital Foundation, Inc.|Scheduling of multiple files for serving on a server|

---

DE20020251U1|2000-11-29|2001-02-22|Autoliv Dev|Force limiter retractor with matching webbing|

---

US7240358B2|2000-12-08|2007-07-03|Digital Fountain, Inc.|Methods and apparatus for scheduling, serving, receiving media-on demand for clients, servers arranged according to constraints on resources|

---

AT464740T|2000-12-15|2010-04-15|British Telecomm|TRANSFER OF SOUND AND / OR PICTURE MATERIAL|

---

AU2092702A|2000-12-15|2002-06-24|British Telecomm|Transmission and reception of audio and/or video material|

---

US6850736B2|2000-12-21|2005-02-01|Tropian, Inc.|Method and apparatus for reception quality indication in wireless communication|

---

US7143433B1|2000-12-27|2006-11-28|Infovalve Computing Inc.|Video distribution system using dynamic segmenting of video data files|

---

US20020085013A1|2000-12-29|2002-07-04|Lippincott Louis A.|Scan synchronized dual frame buffer graphics subsystem|

---

NO315887B1|2001-01-04|2003-11-03|Fast Search & Transfer As|Procedures for transmitting and socking video information|

---

US20080059532A1|2001-01-18|2008-03-06|Kazmi Syed N|Method and system for managing digital content, including streaming media|

---

KR100674423B1|2001-01-19|2007-01-29|엘지전자 주식회사|Transmitting/receiving system and data processing method|

---

DE10103387A1|2001-01-26|2002-08-01|Thorsten Nordhoff|Wind power plant with a device for obstacle lighting or night marking|

---

FI118830B|2001-02-08|2008-03-31|Nokia Corp|Streaming playback|

---

US6868083B2|2001-02-16|2005-03-15|Hewlett-Packard Development Company, L.P.|Method and system for packet communication employing path diversity|

---

WO2002071640A1|2001-03-05|2002-09-12|Intervideo, Inc.|Systems and methods for encoding and decoding redundant motion vectors in compressed video bitstreams|

---

US20020129159A1|2001-03-09|2002-09-12|Michael Luby|Multi-output packet server with independent streams|

---

KR100464360B1|2001-03-30|2005-01-03|삼성전자주식회사|Apparatus and method for efficiently energy distributing over packet data channel in mobile communication system for high rate packet transmission|

---

TWI246841B|2001-04-22|2006-01-01|Koninkl Philips Electronics Nv|Digital transmission system and method for transmitting digital signals|

---

US20020143953A1|2001-04-03|2002-10-03|International Business Machines Corporation|Automatic affinity within networks performing workload balancing|

---

US6785836B2|2001-04-11|2004-08-31|Broadcom Corporation|In-place data transformation for fault-tolerant disk storage systems|

---

US6820221B2|2001-04-13|2004-11-16|Hewlett-Packard Development Company, L.P.|System and method for detecting process and network failures in a distributed system|

---

US7010052B2|2001-04-16|2006-03-07|The Ohio University|Apparatus and method of CTCM encoding and decoding for a digital communication system|

---

US7035468B2|2001-04-20|2006-04-25|Front Porch Digital Inc.|Methods and apparatus for archiving, indexing and accessing audio and video data|

---

US20020191116A1|2001-04-24|2002-12-19|Damien Kessler|System and data format for providing seamless stream switching in a digital video recorder|

---

US20020194608A1|2001-04-26|2002-12-19|Goldhor Richard S.|Method and apparatus for a playback enhancement system implementing a "Say Again" feature|

---

US6497479B1|2001-04-27|2002-12-24|Hewlett-Packard Company|Higher organic inks with good reliability and drytime|

---

US7962482B2|2001-05-16|2011-06-14|Pandora Media, Inc.|Methods and systems for utilizing contextual feedback to generate and modify playlists|

---

US6633856B2|2001-06-15|2003-10-14|Flarion Technologies, Inc.|Methods and apparatus for decoding LDPC codes|

---

US7076478B2|2001-06-26|2006-07-11|Microsoft Corporation|Wrapper playlists on streaming media services|

---

US6745364B2|2001-06-28|2004-06-01|Microsoft Corporation|Negotiated/dynamic error correction for streamed media|

---

JP2003018568A|2001-06-29|2003-01-17|Matsushita Electric Ind Co Ltd|Reproducing system, server apparatus and reproducer|

---

JP2003022232A|2001-07-06|2003-01-24|Fujitsu Ltd|Contents data transferring system|

---

US6895547B2|2001-07-11|2005-05-17|International Business Machines Corporation|Method and apparatus for low density parity check encoding of data|

---

US6928603B1|2001-07-19|2005-08-09|Adaptix, Inc.|System and method for interference mitigation using adaptive forward error correction in a wireless RF data transmission system|

---

US6961890B2|2001-08-16|2005-11-01|Hewlett-Packard Development Company, L.P.|Dynamic variable-length error correction code|

---

US7110412B2|2001-09-18|2006-09-19|Sbc Technology Resources, Inc.|Method and system to transport high-quality video signals|

---

FI115418B|2001-09-20|2005-04-29|Oplayo Oy|Adaptive media stream|

---

US6990624B2|2001-10-12|2006-01-24|Agere Systems Inc.|High speed syndrome-based FEC encoder and decoder and system using same|

---

US7480703B2|2001-11-09|2009-01-20|Sony Corporation|System, method, and computer program product for remotely determining the configuration of a multi-media content user based on response of the user|

---

US7363354B2|2001-11-29|2008-04-22|Nokia Corporation|System and method for identifying and accessing network services|

---

US7003712B2|2001-11-29|2006-02-21|Emin Martinian|Apparatus and method for adaptive, multimode decoding|

---

JP2003174489A|2001-12-05|2003-06-20|Ntt Docomo Inc|Streaming distribution device and streaming distribution method|

---

US7068729B2|2001-12-21|2006-06-27|Digital Fountain, Inc.|Multi-stage code generator and decoder for communication systems|

---

FI114527B|2002-01-23|2004-10-29|Nokia Corp|Grouping of picture frames in video encoding|

---

KR100959573B1|2002-01-23|2010-05-27|노키아 코포레이션|Grouping of image frames in video coding|

---

CN1625880B|2002-01-30|2010-08-11|Nxp股份有限公司|Streaming multimedia data over a network having a variable bandwith|

---

WO2003071440A1|2002-02-15|2003-08-28|Digital Fountain, Inc.|System and method for reliably communicating the content of a live data stream|

---

JP4126928B2|2002-02-28|2008-07-30|日本電気株式会社|Proxy server and proxy control program|

---

JP4116470B2|2002-03-06|2008-07-09|ヒューレット・パッカード・カンパニー|Media streaming distribution system|

---

FR2837332A1|2002-03-15|2003-09-19|Thomson Licensing Sa|DEVICE AND METHOD FOR INSERTING ERROR CORRECTION AND RECONSTITUTION CODES OF DATA STREAMS, AND CORRESPONDING PRODUCTS|

---

MXPA04010058A|2002-04-15|2004-12-13|Nokia Corp|Rlp logical layer of a communication station.|

---

US6677864B2|2002-04-18|2004-01-13|Telefonaktiebolaget L.M. Ericsson|Method for multicast over wireless networks|

---

JP3629008B2|2002-04-19|2005-03-16|松下電器産業株式会社|Data receiving apparatus and data distribution system|

---

JP3689063B2|2002-04-19|2005-08-31|松下電器産業株式会社|Data receiving apparatus and data distribution system|

---

WO2003092305A1|2002-04-25|2003-11-06|Sharp Kabushiki Kaisha|Image encodder, image decoder, record medium, and image recorder|

---

US20030204602A1|2002-04-26|2003-10-30|Hudson Michael D.|Mediated multi-source peer content delivery network architecture|

---

US7177658B2|2002-05-06|2007-02-13|Qualcomm, Incorporated|Multi-media broadcast and multicast service in a wireless communications system|

---

US7200388B2|2002-05-31|2007-04-03|Nokia Corporation|Fragmented delivery of multimedia|

---

US20040083015A1|2002-06-04|2004-04-29|Srinivas Patwari|System for multimedia rendering in a portable device|

---

KR20050010851A|2002-06-04|2005-01-28|퀄컴 인코포레이티드|System for multimedia rendering in a portable device|

---

US9419749B2|2009-08-19|2016-08-16|Qualcomm Incorporated|Methods and apparatus employing FEC codes with permanent inactivation of symbols for encoding and decoding processes|

---

ES2445761T3|2002-06-11|2014-03-05|Digital Fountain, Inc.|Decoding chain reaction codes by inactivation|

---

US9240810B2|2002-06-11|2016-01-19|Digital Fountain, Inc.|Systems and processes for decoding chain reaction codes through inactivation|

---

WO2003105484A1|2002-06-11|2003-12-18|Telefonaktiebolaget L M Ericsson |Generation of mixed media streams|

---

US9288010B2|2009-08-19|2016-03-15|Qualcomm Incorporated|Universal file delivery methods for providing unequal error protection and bundled file delivery services|

---

US6956875B2|2002-06-19|2005-10-18|Atlinks Usa, Inc.|Technique for communicating variable bit rate data over a constant bit rate link|

---

JP4154569B2|2002-07-10|2008-09-24|日本電気株式会社|Image compression / decompression device|

---

JP4120461B2|2002-07-12|2008-07-16|住友電気工業株式会社|Transmission data generation method and transmission data generation apparatus|

---

KR100754419B1|2002-07-16|2007-08-31|노키아 코포레이션|A method for random access and gradual picture refresh in video coding|

---

US7664126B2|2002-07-31|2010-02-16|Sharp Kabushiki Kaisha|Data communication apparatus, intermittent communication method therefor, program describing the method and recording medium for recording the program|

---

JP2004070712A|2002-08-07|2004-03-04|Nippon Telegr & Teleph Corp <Ntt>|Data delivery method, data delivery system, split delivery data receiving method, split delivery data receiving device and split delivery data receiving program|

---

AU2002319335B2|2002-08-13|2008-12-04|Nokia Corporation|Symbol interleaving|

---

US6985459B2|2002-08-21|2006-01-10|Qualcomm Incorporated|Early transmission and playout of packets in wireless communication systems|

---

JP2004112129A|2002-09-17|2004-04-08|Toshiba Corp|Image delivering apparatus and program for realizing image delivering step|

---

WO2004030273A1|2002-09-27|2004-04-08|Fujitsu Limited|Data delivery method, system, transfer method, and program|

---

JP3534742B1|2002-10-03|2004-06-07|株式会社エヌ・ティ・ティ・ドコモ|Moving picture decoding method, moving picture decoding apparatus, and moving picture decoding program|

---

AU2003277198A1|2002-10-05|2004-05-04|Digital Fountain, Inc.|Systematic encoding and decoding of chain reaction codes|

---

JP2004135013A|2002-10-10|2004-04-30|Matsushita Electric Ind Co Ltd|Device and method for transmission|

---

FI116816B|2002-10-14|2006-02-28|Nokia Corp|Streaming media|

---

US8320301B2|2002-10-25|2012-11-27|Qualcomm Incorporated|MIMO WLAN system|

---

US7289451B2|2002-10-25|2007-10-30|Telefonaktiebolaget Lm Ericsson |Delay trading between communication links|

---

WO2004040831A1|2002-10-30|2004-05-13|Koninklijke Philips Electronics N.V.|Adaptative forward error control scheme|

---

JP2004165922A|2002-11-12|2004-06-10|Sony Corp|Apparatus, method, and program for information processing|

---

GB0226872D0|2002-11-18|2002-12-24|British Telecomm|Video transmission|

---

WO2004047455A1|2002-11-18|2004-06-03|British Telecommunications Public Limited Company|Transmission of video|

---

JP3935419B2|2002-11-19|2007-06-20|Ｋｄｄｉ株式会社|Video coding bit rate selection method|

---

KR100502609B1|2002-11-21|2005-07-20|한국전자통신연구원|Encoder using low density parity check code and encoding method thereof|

---

US7086718B2|2002-11-23|2006-08-08|Silverbrook Research Pty Ltd|Thermal ink jet printhead with high nozzle areal density|

---

JP2004192140A|2002-12-09|2004-07-08|Sony Corp|Data communication system, data transmitting device, data receiving device and method, and computer program|

---

JP2004193992A|2002-12-11|2004-07-08|Sony Corp|Information processing system, information processor, information processing method, recording medium and program|

---

US8135073B2|2002-12-19|2012-03-13|Trident Microsystems Ltd|Enhancing video images depending on prior image enhancements|

---

US7164882B2|2002-12-24|2007-01-16|Poltorak Alexander I|Apparatus and method for facilitating a purchase using information provided on a media playing device|

---

WO2004068715A2|2003-01-29|2004-08-12|Digital Fountain, Inc.|Systems and processes for fast encoding of hamming codes|

---

US7525994B2|2003-01-30|2009-04-28|Avaya Inc.|Packet data flow identification for multiplexing|

---

US7756002B2|2003-01-30|2010-07-13|Texas Instruments Incorporated|Time-frequency interleaved orthogonal frequency division multiplexing ultra wide band physical layer|

---

US7231404B2|2003-01-31|2007-06-12|Nokia Corporation|Datacast file transmission with meta-data retention|

---

US7062272B2|2003-02-18|2006-06-13|Qualcomm Incorporated|Method and apparatus to track count of broadcast content recipients in a wireless telephone network|

---

EP1455504B1|2003-03-07|2014-11-12|Samsung Electronics Co., Ltd.|Apparatus and method for processing audio signal and computer readable recording medium storing computer program for the method|

---

JP4173755B2|2003-03-24|2008-10-29|富士通株式会社|Data transmission server|

---

US7610487B2|2003-03-27|2009-10-27|Microsoft Corporation|Human input security codes|

---

US7266147B2|2003-03-31|2007-09-04|Sharp Laboratories Of America, Inc.|Hypothetical reference decoder|

---

JP2004343701A|2003-04-21|2004-12-02|Matsushita Electric Ind Co Ltd|Data receiving reproduction apparatus, data receiving reproduction method, and data receiving reproduction processing program|

---

US7408486B2|2003-04-21|2008-08-05|Qbit Corporation|System and method for using a microlet-based modem|

---

JP4379779B2|2003-04-28|2009-12-09|Ｋｄｄｉ株式会社|Video distribution method|

---

US20050041736A1|2003-05-07|2005-02-24|Bernie Butler-Smith|Stereoscopic television signal processing method, transmission system and viewer enhancements|

---

KR100492567B1|2003-05-13|2005-06-03|엘지전자 주식회사|Http-based video streaming apparatus and method for a mobile communication system|

---

US7113773B2|2003-05-16|2006-09-26|Qualcomm Incorporated|Reliable reception of broadcast/multicast content|

---

JP2004348824A|2003-05-21|2004-12-09|Toshiba Corp|Ecc encoding method and ecc encoding device|

---

US7483525B2|2003-05-23|2009-01-27|Navin Chaddha|Method and system for selecting a communication channel with a recipient device over a communication network|

---

JP2004362099A|2003-06-03|2004-12-24|Sony Corp|Server device, information processor, information processing method, and computer program|

---

MXPA05013237A|2003-06-07|2006-03-09|Samsung Electronics Co Ltd|Apparatus and method for organization and interpretation of multimedia data on a recording medium.|

---

KR101003413B1|2003-06-12|2010-12-23|엘지전자 주식회사|Method for compression/decompression the transferring data of mobile phone|

---

US7603689B2|2003-06-13|2009-10-13|Microsoft Corporation|Fast start-up for digital video streams|

---

RU2265960C2|2003-06-16|2005-12-10|Федеральное государственное унитарное предприятие "Калужский научно-исследовательский институт телемеханических устройств"|Method for transferring information with use of adaptive alternation|

---

US7391717B2|2003-06-30|2008-06-24|Microsoft Corporation|Streaming of variable bit rate multimedia content|

---

US20050004997A1|2003-07-01|2005-01-06|Nokia Corporation|Progressive downloading of timed multimedia content|

---

US8149939B2|2003-07-07|2012-04-03|Samsung Electronics Co., Ltd.|System of robust DTV signal transmissions that legacy DTV receivers will disregard|

---

US7254754B2|2003-07-14|2007-08-07|International Business Machines Corporation|Raid 3+3|

---

KR100532450B1|2003-07-16|2005-11-30|삼성전자주식회사|Data recording method with robustness for errors, data reproducing method therefore, and apparatuses therefore|

---

US20050028067A1|2003-07-31|2005-02-03|Weirauch Charles R.|Data with multiple sets of error correction codes|

---

US8694869B2|2003-08-21|2014-04-08|QUALCIMM Incorporated|Methods for forward error correction coding above a radio link control layer and related apparatus|

---

CN1868157B|2003-08-21|2011-07-27|高通股份有限公司|Methods for forward error correction coding above a radio link control layer and related apparatus|

---

IL157885D0|2003-09-11|2004-03-28|Bamboo Mediacasting Ltd|Iterative forward error correction|

---

IL157886D0|2003-09-11|2009-02-11|Bamboo Mediacasting Ltd|Secure multicast transmission|

---

JP4183586B2|2003-09-12|2008-11-19|三洋電機株式会社|Video display device|

---

JP4988346B2|2003-09-15|2012-08-01|ザ・ディレクティービー・グループ・インコーポレイテッド|Method and system for adaptive transcoding and rate conversion in video networks|

---

KR100608715B1|2003-09-27|2006-08-04|엘지전자 주식회사|SYSTEM AND METHOD FOR QoS-QUARANTED MULTIMEDIA STREAMING SERVICE|

---

EP1521373B1|2003-09-30|2006-08-23|Telefonaktiebolaget LM Ericsson |In-place data deinterleaving|

---

US7559004B1|2003-10-01|2009-07-07|Sandisk Corporation|Dynamic redundant area configuration in a non-volatile memory system|

---

EP2722995A3|2003-10-06|2018-01-17|Digital Fountain, Inc.|Soft-decision decoding of multi-stage chain reaction codes|

---

US7516232B2|2003-10-10|2009-04-07|Microsoft Corporation|Media organization for distributed sending of media data|

---

US7614071B2|2003-10-10|2009-11-03|Microsoft Corporation|Architecture for distributed sending of media data|

---

CN100555213C|2003-10-14|2009-10-28|松下电器产业株式会社|Data converter|

---

US7650036B2|2003-10-16|2010-01-19|Sharp Laboratories Of America, Inc.|System and method for three-dimensional video coding|

---

US7168030B2|2003-10-17|2007-01-23|Telefonaktiebolaget Lm Ericsson |Turbo code decoder with parity information update|

---

US8132215B2|2003-10-27|2012-03-06|Panasonic Corporation|Apparatus for receiving broadcast signal|

---

JP2005136546A|2003-10-29|2005-05-26|Sony Corp|Transmission apparatus and method, recording medium, and program|

---

EP1528702B1|2003-11-03|2008-01-23|Broadcom Corporation|FEC decoding with dynamic parameters|

---

US20050102371A1|2003-11-07|2005-05-12|Emre Aksu|Streaming from a server to a client|

---

WO2005055016A2|2003-12-01|2005-06-16|Digital Fountain, Inc.|Protection of data from erasures using subsymbol based codes|

---

US7428669B2|2003-12-07|2008-09-23|Adaptive Spectrum And Signal Alignment, Inc.|Adaptive FEC codeword management|

---

US7574706B2|2003-12-15|2009-08-11|Microsoft Corporation|System and method for managing and communicating software updates|

---

US7590118B2|2003-12-23|2009-09-15|Agere Systems Inc.|Frame aggregation format|

---

JP4536383B2|2004-01-16|2010-09-01|株式会社エヌ・ティ・ティ・ドコモ|Data receiving apparatus and data receiving method|

---

KR100770902B1|2004-01-20|2007-10-26|삼성전자주식회사|Apparatus and method for generating and decoding forward error correction codes of variable rate by using high rate data wireless communication|

---

KR100834750B1|2004-01-29|2008-06-05|삼성전자주식회사|Appartus and method for Scalable video coding providing scalability in encoder part|

---

JP4321284B2|2004-02-03|2009-08-26|株式会社デンソー|Streaming data transmission apparatus and information distribution system|

---

US7599294B2|2004-02-13|2009-10-06|Nokia Corporation|Identification and re-transmission of missing parts|

---

KR100596705B1|2004-03-04|2006-07-04|삼성전자주식회사|Method and system for video coding for video streaming service, and method and system for video decoding|

---

KR100586883B1|2004-03-04|2006-06-08|삼성전자주식회사|Method and apparatus for video coding, pre-decoding, video decoding for vidoe streaming service, and method for image filtering|

---

US7609653B2|2004-03-08|2009-10-27|Microsoft Corporation|Resolving partial media topologies|

---

US20050207392A1|2004-03-19|2005-09-22|Telefonaktiebolaget Lm Ericsson |Higher layer packet framing using RLP|

---

US7240236B2|2004-03-23|2007-07-03|Archivas, Inc.|Fixed content distributed data storage using permutation ring encoding|

---

US7930184B2|2004-08-04|2011-04-19|Dts, Inc.|Multi-channel audio coding/decoding of random access points and transients|

---

JP4433287B2|2004-03-25|2010-03-17|ソニー株式会社|Receiving apparatus and method, and program|

---

US8842175B2|2004-03-26|2014-09-23|Broadcom Corporation|Anticipatory video signal reception and processing|

---

US20050216472A1|2004-03-29|2005-09-29|David Leon|Efficient multicast/broadcast distribution of formatted data|

---

KR20070007810A|2004-03-30|2007-01-16|코닌클리케 필립스 일렉트로닉스 엔.브이.|System and method for supporting improved trick mode performance for disc-based multimedia content|

---

TW200534875A|2004-04-23|2005-11-01|Lonza Ag|Personal care compositions and concentrates for making the same|

---

FR2869744A1|2004-04-29|2005-11-04|Thomson Licensing Sa|METHOD FOR TRANSMITTING DIGITAL DATA PACKETS AND APPARATUS IMPLEMENTING THE METHOD|

---

US8868772B2|2004-04-30|2014-10-21|Echostar Technologies L.L.C.|Apparatus, system, and method for adaptive-rate shifting of streaming content|

---

US7633970B2|2004-05-07|2009-12-15|Agere Systems Inc.|MAC header compression for use with frame aggregation|

---

EP1743431A4|2004-05-07|2007-05-02|Digital Fountain Inc|File download and streaming system|

---

US20050254575A1|2004-05-12|2005-11-17|Nokia Corporation|Multiple interoperability points for scalable media coding and transmission|

---

US20050254526A1|2004-05-12|2005-11-17|Nokia Corporation|Parameter sets update in streaming applications|

---

US20060037057A1|2004-05-24|2006-02-16|Sharp Laboratories Of America, Inc.|Method and system of enabling trick play modes using HTTP GET|

---

US8331445B2|2004-06-01|2012-12-11|Qualcomm Incorporated|Method, apparatus, and system for enhancing robustness of predictive video codecs using a side-channel based on distributed source coding techniques|

---

US20070110074A1|2004-06-04|2007-05-17|Bob Bradley|System and Method for Synchronizing Media Presentation at Multiple Recipients|

---

US7492828B2|2004-06-18|2009-02-17|Qualcomm Incorporated|Time synchronization using spectral estimation in a communication system|

---

US8112531B2|2004-07-14|2012-02-07|Nokia Corporation|Grouping of session objects|

---

US7139660B2|2004-07-14|2006-11-21|General Motors Corporation|System and method for changing motor vehicle personalization settings|

---

US8544043B2|2004-07-21|2013-09-24|Qualcomm Incorporated|Methods and apparatus for providing content information to content servers|

---

JP2006033763A|2004-07-21|2006-02-02|Toshiba Corp|Electronic apparatus and communication control method|

---

US7409626B1|2004-07-28|2008-08-05|Ikanos Communications Inc|Method and apparatus for determining codeword interleaver parameters|

---

US7376150B2|2004-07-30|2008-05-20|Nokia Corporation|Point-to-point repair response mechanism for point-to-multipoint transmission systems|

---

US7590922B2|2004-07-30|2009-09-15|Nokia Corporation|Point-to-point repair request mechanism for point-to-multipoint transmission systems|

---

WO2006020826A2|2004-08-11|2006-02-23|Digital Fountain, Inc.|Method and apparatus for fast encoding of data symbols according to half-weight codes|

---

JP4405875B2|2004-08-25|2010-01-27|富士通株式会社|Method and apparatus for generating data for error correction, generation program, and computer-readable recording medium storing the program|

---

JP2006074335A|2004-09-01|2006-03-16|Nippon Telegr & Teleph Corp <Ntt>|Transmission method, transmission system, and transmitter|

---

JP4576936B2|2004-09-02|2010-11-10|ソニー株式会社|Information processing apparatus, information recording medium, content management system, data processing method, and computer program|

---

EP1638333A1|2004-09-17|2006-03-22|Mitsubishi Electric Information Technology Centre Europe B.V.|Rate adaptive video coding|

---

JP2006115104A|2004-10-13|2006-04-27|Daiichikosho Co Ltd|Method and device for packetizing time-series information encoded with high efficiency, and performing real-time streaming transmission, and for reception and reproduction|

---

US7529984B2|2004-11-16|2009-05-05|Infineon Technologies Ag|Seamless change of depth of a general convolutional interleaver during transmission without loss of data|

---

US7751324B2|2004-11-19|2010-07-06|Nokia Corporation|Packet stream arrangement in multimedia transmission|

---

US20080196061A1|2004-11-22|2008-08-14|Boyce Jill Macdonald|Method and Apparatus for Channel Change in Dsl System|

---

JP5425397B2|2004-12-02|2014-02-26|トムソンライセンシング|Apparatus and method for adaptive forward error correction|

---

KR20060065482A|2004-12-10|2006-06-14|마이크로소프트 코포레이션|A system and process for controlling the coding bit rate of streaming media data|

---

JP2006174032A|2004-12-15|2006-06-29|Sanyo Electric Co Ltd|Image data transmission system, image data receiver and image data transmitter|

---

JP2006174045A|2004-12-15|2006-06-29|Ntt Communications Kk|Image distribution device, program, and method therefor|

---

US7398454B2|2004-12-21|2008-07-08|Tyco Telecommunications Inc.|System and method for forward error correction decoding using soft information|

---

JP4391409B2|2004-12-24|2009-12-24|株式会社第一興商|High-efficiency-encoded time-series information transmission method and apparatus for real-time streaming transmission and reception|

---

WO2006084503A1|2005-02-08|2006-08-17|Telefonaktiebolaget Lm Ericsson |On-demand multi-channel streaming session over packet-switched networks|

---

US7925097B2|2005-02-18|2011-04-12|Sanyo Electric Co., Ltd.|Image display method, image coding apparatus, and image decoding apparatus|

---

US7822139B2|2005-03-02|2010-10-26|Rohde & Schwarz Gmbh & Co. Kg|Apparatus, systems, methods and computer products for providing a virtual enhanced training sequence|

---

WO2006096104A1|2005-03-07|2006-09-14|Telefonaktiebolaget Lm Ericsson |Multimedia channel switching|

---

US8028322B2|2005-03-14|2011-09-27|Time Warner Cable Inc.|Method and apparatus for network content download and recording|

---

US7219289B2|2005-03-15|2007-05-15|Tandberg Data Corporation|Multiply redundant raid system and XOR-efficient method and apparatus for implementing the same|

---

US7418649B2|2005-03-15|2008-08-26|Microsoft Corporation|Efficient implementation of reed-solomon erasure resilient codes in high-rate applications|

---

US7450064B2|2005-03-22|2008-11-11|Qualcomm, Incorporated|Methods and systems for deriving seed position of a subscriber station in support of unassisted GPS-type position determination in a wireless communication system|

---

JP4487028B2|2005-03-31|2010-06-23|ブラザー工業株式会社|Delivery speed control device, delivery system, delivery speed control method, and delivery speed control program|

---

US7715842B2|2005-04-09|2010-05-11|Lg Electronics Inc.|Supporting handover of mobile terminal|

---

WO2006108917A1|2005-04-13|2006-10-19|Nokia Corporation|Coding, storage and signalling of scalability information|

---

KR100643291B1|2005-04-14|2006-11-10|삼성전자주식회사|Apparatus and method of video encoding and decoding for minimizing random access delay|

---

JP4515319B2|2005-04-27|2010-07-28|株式会社日立製作所|Computer system|

---

US7961700B2|2005-04-28|2011-06-14|Qualcomm Incorporated|Multi-carrier operation in data transmission systems|

---

JP2006319743A|2005-05-13|2006-11-24|Toshiba Corp|Receiving device|

---

CA2562212C|2005-10-05|2012-07-10|Lg Electronics Inc.|Method of processing traffic information and digital broadcast system|

---

US8228994B2|2005-05-20|2012-07-24|Microsoft Corporation|Multi-view video coding based on temporal and view decomposition|

---

JP2008543142A|2005-05-24|2008-11-27|ノキアコーポレイション|Method and apparatus for hierarchical transmission and reception in digital broadcasting|

---

US7676735B2|2005-06-10|2010-03-09|Digital Fountain Inc.|Forward error-correcting coding and streaming|

---

US9386064B2|2006-06-09|2016-07-05|Qualcomm Incorporated|Enhanced block-request streaming using URL templates and construction rules|

---

US9178535B2|2006-06-09|2015-11-03|Digital Fountain, Inc.|Dynamic stream interleaving and sub-stream based delivery|

---

US7644335B2|2005-06-10|2010-01-05|Qualcomm Incorporated|In-place transformations with applications to encoding and decoding various classes of codes|

---

US9209934B2|2006-06-09|2015-12-08|Qualcomm Incorporated|Enhanced block-request streaming using cooperative parallel HTTP and forward error correction|

---

US9380096B2|2006-06-09|2016-06-28|Qualcomm Incorporated|Enhanced block-request streaming system for handling low-latency streaming|

---

JP2007013436A|2005-06-29|2007-01-18|Toshiba Corp|Coding stream reproducing apparatus|

---

US20070006274A1|2005-06-30|2007-01-04|Toni Paila|Transmission and reception of session packets|

---

JP2007013675A|2005-06-30|2007-01-18|Sanyo Electric Co Ltd|Streaming distribution system and server|

---

US7725593B2|2005-07-15|2010-05-25|Sony Corporation|Scalable video coding file format|

---

US20070022215A1|2005-07-19|2007-01-25|Singer David W|Method and apparatus for media data transmission|

---

JP2007036666A|2005-07-27|2007-02-08|Onkyo Corp|Contents distribution system, client, and client program|

---

AT514246T|2005-08-19|2011-07-15|Hewlett Packard Development Co|STATEMENT OF LOST SEGMENTS ON LAYER LIMITS|

---

CN101053249B|2005-09-09|2011-02-16|松下电器产业株式会社|Image processing method, image storage method, image processing device and image file format|

---

US7924913B2|2005-09-15|2011-04-12|Microsoft Corporation|Non-realtime data transcoding of multimedia content|

---

US20070067480A1|2005-09-19|2007-03-22|Sharp Laboratories Of America, Inc.|Adaptive media playout by server media processing for robust streaming|

---

US9113147B2|2005-09-27|2015-08-18|Qualcomm Incorporated|Scalability techniques based on content information|

---

US20070078876A1|2005-09-30|2007-04-05|Yahoo! Inc.|Generating a stream of media data containing portions of media files using location tags|

---

US7164370B1|2005-10-06|2007-01-16|Analog Devices, Inc.|System and method for decoding data compressed in accordance with dictionary-based compression schemes|

---

CN100442858C|2005-10-11|2008-12-10|华为技术有限公司|Lip synchronous method for multimedia real-time transmission in packet network and apparatus thereof|

---

EP1935182B1|2005-10-11|2016-11-23|Nokia Technologies Oy|System and method for efficient scalable stream adaptation|

---

US7720096B2|2005-10-13|2010-05-18|Microsoft Corporation|RTP payload format for VC-1|

---

US8654848B2|2005-10-17|2014-02-18|Qualcomm Incorporated|Method and apparatus for shot detection in video streaming|

---

EP1946563A2|2005-10-19|2008-07-23|Thomson Licensing|Multi-view video coding using scalable video coding|

---

JP4727401B2|2005-12-02|2011-07-20|日本電信電話株式会社|Wireless multicast transmission system, wireless transmission device, and wireless multicast transmission method|

---

FR2894421B1|2005-12-07|2008-01-18|Canon Kk|METHOD AND DEVICE FOR DECODING A VIDEO STREAM CODE FOLLOWING A HIERARCHICAL CODING|

---

KR100759823B1|2005-12-08|2007-09-18|한국전자통신연구원|Apparatus for generating RZreturn to zero signal and method thereof|

---

JP4456064B2|2005-12-21|2010-04-28|日本電信電話株式会社|Packet transmission device, reception device, system, and program|

---

US20070157267A1|2005-12-30|2007-07-05|Intel Corporation|Techniques to improve time seek operations|

---

KR101353620B1|2006-01-05|2014-01-20|텔레폰악티에볼라겟엘엠에릭슨|Media container file management|

---

US8214516B2|2006-01-06|2012-07-03|Google Inc.|Dynamic media serving infrastructure|

---

BRPI0707457A2|2006-01-11|2011-05-03|Nokia Corp|inverse compatible aggregation of images in resizable video encoding|

---

KR100947234B1|2006-01-12|2010-03-12|엘지전자 주식회사|Method and apparatus for processing multiview video|

---

WO2007086654A1|2006-01-25|2007-08-02|Lg Electronics Inc.|Digital broadcasting system and method of processing data|

---

US7262719B2|2006-01-30|2007-08-28|International Business Machines Corporation|Fast data stream decoding using apriori information|

---

RU2290768C1|2006-01-30|2006-12-27|Общество с ограниченной ответственностью "Трафиклэнд"|Media broadcast system in infrastructure of mobile communications operator|

---

GB0602314D0|2006-02-06|2006-03-15|Ericsson Telefon Ab L M|Transporting packets|

---

US20110087792A2|2006-02-07|2011-04-14|Dot Hill Systems Corporation|Data replication method and apparatus|

---

US8239727B2|2006-02-08|2012-08-07|Thomson Licensing|Decoding of raptor codes|

---

KR101292851B1|2006-02-13|2013-08-02|디지털 파운튼, 인크.|Streaming and buffering using variable fec overhead and protection periods|

---

US20070200949A1|2006-02-21|2007-08-30|Qualcomm Incorporated|Rapid tuning in multimedia applications|

---

US9270414B2|2006-02-21|2016-02-23|Digital Fountain, Inc.|Multiple-field based code generator and decoder for communications systems|

---

JP2007228205A|2006-02-23|2007-09-06|Funai Electric Co Ltd|Network server|

---

US8320450B2|2006-03-29|2012-11-27|Vidyo, Inc.|System and method for transcoding between scalable and non-scalable video codecs|

---

US20080010153A1|2006-04-24|2008-01-10|Pugh-O'connor Archie|Computer network provided digital content under an advertising and revenue sharing basis, such as music provided via the internet with time-shifted advertisements presented by a client resident application|

---

WO2007127741A2|2006-04-24|2007-11-08|Sun Microsystems, Inc.|Media server system|

---

US7640353B2|2006-04-27|2009-12-29|Microsoft Corporation|Guided random seek support for media streaming|

---

US7971129B2|2006-05-10|2011-06-28|Digital Fountain, Inc.|Code generator and decoder for communications systems operating using hybrid codes to allow for multiple efficient users of the communications systems|

---

US7525993B2|2006-05-24|2009-04-28|Newport Media, Inc.|Robust transmission system and method for mobile television applications|

---

TWM302355U|2006-06-09|2006-12-11|Jia-Bau Jeng|Fixation and cushion structure of knee joint|

---

JP2008011404A|2006-06-30|2008-01-17|Toshiba Corp|Content processing apparatus and method|

---

JP4392004B2|2006-07-03|2009-12-24|インターナショナル・ビジネス・マシーンズ・コーポレーション|Encoding and decoding techniques for packet recovery|

---

EP2302869A3|2006-07-20|2013-05-22|SanDisk Technologies Inc.|An improved audio visual player apparatus and system and method of content distribution using the same|

---

GB0614891D0|2006-07-27|2006-09-06|Univ Southampton|Plasmon-enhanced photo voltaic cell|

---

US7711797B1|2006-07-31|2010-05-04|Juniper Networks, Inc.|Optimizing batch size for prefetching data over wide area networks|

---

US8209736B2|2006-08-23|2012-06-26|Mediatek Inc.|Systems and methods for managing television signals|

---

US20080066136A1|2006-08-24|2008-03-13|International Business Machines Corporation|System and method for detecting topic shift boundaries in multimedia streams using joint audio, visual and text cues|

---

EP2055107B1|2006-08-24|2013-05-15|Nokia Corporation|Hint of tracks relationships for multi-stream media files in multiple description coding MDC.|

---

JP2008109637A|2006-09-25|2008-05-08|Toshiba Corp|Motion picture encoding apparatus and method|

---

US8015556B2|2006-10-12|2011-09-06|International Business Machines Corporation|Efficient method of data reshaping for multidimensional dynamic array objects in the presence of multiple object instantiations|

---

US8428013B2|2006-10-30|2013-04-23|Lg Electronics Inc.|Method of performing random access in a wireless communcation system|

---

JP2008118221A|2006-10-31|2008-05-22|Toshiba Corp|Decoder and decoding method|

---

WO2008054100A1|2006-11-01|2008-05-08|Electronics And Telecommunications Research Institute|Method and apparatus for decoding metadata used for playing stereoscopic contents|

---

MX2009005086A|2006-11-14|2009-05-27|Qualcomm Inc|Systems and methods for channel switching.|

---

US8027328B2|2006-12-26|2011-09-27|Alcatel Lucent|Header compression in a wireless communication network|

---

WO2008086313A1|2007-01-05|2008-07-17|Divx, Inc.|Video distribution system including progressive playback|

---

US20080168516A1|2007-01-08|2008-07-10|Christopher Lance Flick|Facilitating Random Access In Streaming Content|

---

WO2008084348A1|2007-01-09|2008-07-17|Nokia Corporation|Method for supporting file versioning in mbms file repair|

---

US20080172430A1|2007-01-11|2008-07-17|Andrew Thomas Thorstensen|Fragmentation Compression Management|

---

WO2008084876A1|2007-01-11|2008-07-17|Panasonic Corporation|Method for trick playing on streamed and encrypted multimedia|

---

EP3484123A1|2007-01-12|2019-05-15|University-Industry Cooperation Group Of Kyung Hee University|Packet format of network abstraction layer unit, and algorithm and apparatus for video encoding and decoding using the format|

---

KR20080066408A|2007-01-12|2008-07-16|삼성전자주식회사|Device and method for generating three-dimension image and displaying thereof|

---

US8135071B2|2007-01-16|2012-03-13|Cisco Technology, Inc.|Breakpoint determining for hybrid variable length coding using relationship to neighboring blocks|

---

US7721003B2|2007-02-02|2010-05-18|International Business Machines Corporation|System and method to synchronize OSGi bundle inventories between an OSGi bundle server and a client|

---

US7805456B2|2007-02-05|2010-09-28|Microsoft Corporation|Query pattern to enable type flow of element types|

---

US20080192818A1|2007-02-09|2008-08-14|Dipietro Donald Vincent|Systems and methods for securing media|

---

US20080232357A1|2007-03-19|2008-09-25|Legend Silicon Corp.|Ls digital fountain code|

---

JP4838191B2|2007-05-08|2011-12-14|シャープ株式会社|File reproduction device, file reproduction method, program for executing file reproduction, and recording medium recording the program|

---

JP2008283571A|2007-05-11|2008-11-20|Ntt Docomo Inc|Content distribution device, system and method|

---

WO2008140261A2|2007-05-14|2008-11-20|Samsung Electronics Co., Ltd.|Broadcasting service transmitting apparatus and method and broadcasting service receiving apparatus and method for effectively accessing broadcasting service|

---

BRPI0811117A2|2007-05-16|2014-12-23|Thomson Licensing|APPARATUS AND METHOD FOR ENCODING AND DECODING SIGNS|

---

FR2917262A1|2007-06-05|2008-12-12|Thomson Licensing Sas|DEVICE AND METHOD FOR CODING VIDEO CONTENT IN THE FORM OF A SCALABLE FLOW.|

---

US8487982B2|2007-06-07|2013-07-16|Reald Inc.|Stereoplexing for film and video applications|

---

EP2501137A3|2007-06-11|2012-12-12|Samsung Electronics Co., Ltd.|Method and apparatus for generating header information of stereoscopic image|

---

US8340113B2|2007-06-20|2012-12-25|Telefonaktiebolaget Lm Erricsson |Method and arrangement for improved media session management|

---

EP2174502A2|2007-06-26|2010-04-14|Nokia Corporation|System and method for indicating temporal layer switching points|

---

US8706907B2|2007-10-19|2014-04-22|Voxer Ip Llc|Telecommunication and multimedia management method and apparatus|

---

US7917702B2|2007-07-10|2011-03-29|Qualcomm Incorporated|Data prefetch throttle|

---

US8156164B2|2007-07-11|2012-04-10|International Business Machines Corporation|Concurrent directory update in a cluster file system|

---

JP2009027598A|2007-07-23|2009-02-05|Hitachi Ltd|Video distribution server and video distribution method|

---

US8683066B2|2007-08-06|2014-03-25|DISH Digital L.L.C.|Apparatus, system, and method for multi-bitrate content streaming|

---

US8327403B1|2007-09-07|2012-12-04|United Video Properties, Inc.|Systems and methods for providing remote program ordering on a user device via a web server|

---

US9237101B2|2007-09-12|2016-01-12|Digital Fountain, Inc.|Generating and communicating source identification information to enable reliable communications|

---

US8233532B2|2007-09-21|2012-07-31|Fraunhofer-Gesellschaft Zur Foerderung Der Angewandten Forschung E.V.|Information signal, apparatus and method for encoding an information content, and apparatus and method for error correcting an information signal|

---

US8346959B2|2007-09-28|2013-01-01|Sharp Laboratories Of America, Inc.|Client-controlled adaptive streaming|

---

EP2046044B1|2007-10-01|2017-01-18|Cabot Communications Ltd|A method and apparatus for streaming digital media content and a communication system|

---

CN101822021B|2007-10-09|2013-06-05|三星电子株式会社|Apparatus and method for generating and parsing MAC PDU in mobile communication system|

---

US8635360B2|2007-10-19|2014-01-21|Google Inc.|Media playback point seeking using data range requests|

---

US7895629B1|2007-11-07|2011-02-22|At&T Mobility Ii Llc|Video service buffer management in a mobile rate control enabled network|

---

US20090125636A1|2007-11-13|2009-05-14|Qiong Li|Payload allocation methods for scalable multimedia servers|

---

EP2215595B1|2007-11-23|2012-02-22|Media Patents S.L.|A process for the on-line distribution of audiovisual contents with advertisements, advertisement management system, digital rights management system and audiovisual content player provided with said systems|

---

WO2009075766A2|2007-12-05|2009-06-18|Swarmcast, Inc.|Dynamic bit rate scaling|

---

TWI355168B|2007-12-07|2011-12-21|Univ Nat Chiao Tung|Application classification method in network traff|

---

JP5385598B2|2007-12-17|2014-01-08|キヤノン株式会社|Image processing apparatus, image management server apparatus, control method thereof, and program|

---

JP2009152821A|2007-12-20|2009-07-09|Sharp Corp|Digital terrestrial broadcasting receiver|

---

US9313245B2|2007-12-24|2016-04-12|Qualcomm Incorporated|Adaptive streaming for on demand wireless services|

---

KR101506217B1|2008-01-31|2015-03-26|삼성전자주식회사|Method and appratus for generating stereoscopic image data stream for temporally partial three dimensional data, and method and apparatus for displaying temporally partial three dimensional data of stereoscopic image|

---

EP2086237B1|2008-02-04|2012-06-27|Alcatel Lucent|Method and device for reordering and multiplexing multimedia packets from multimedia streams pertaining to interrelated sessions|

---

US8151174B2|2008-02-13|2012-04-03|Sunrise IP, LLC|Block modulus coding systems and methods for block coding with non-binary modulus|

---

US20090219985A1|2008-02-28|2009-09-03|Vasanth Swaminathan|Systems and Methods for Processing Multiple Projections of Video Data in a Single Video File|

---

US7984097B2|2008-03-18|2011-07-19|Media Patents, S.L.|Methods for transmitting multimedia files and advertisements|

---

US8606996B2|2008-03-31|2013-12-10|Amazon Technologies, Inc.|Cache optimization|

---

US20090257508A1|2008-04-10|2009-10-15|Gaurav Aggarwal|Method and system for enabling video trick modes|

---

CN103795511B|2008-04-14|2018-05-01|亚马逊技术股份有限公司|A kind of method that uplink transmission is received in base station and base station|

---

WO2009127961A1|2008-04-16|2009-10-22|Nokia Corporation|Decoding order recovery in session multiplexing|

---

WO2009130561A1|2008-04-21|2009-10-29|Nokia Corporation|Method and device for video coding and decoding|

---

RU2010150108A|2008-05-07|2012-06-20|Диджитал Фаунтин, Инк. |QUICK CHANNEL CHANGE AND HIGH QUALITY STREAM PROTECTION ON A BROADCAST CHANNEL|

---

US7979570B2|2008-05-12|2011-07-12|Swarmcast, Inc.|Live media delivery over a packet-based computer network|

---

JP5022301B2|2008-05-19|2012-09-12|株式会社エヌ・ティ・ティ・ドコモ|Proxy server, communication relay program, and communication relay method|

---

CN101287107B|2008-05-29|2010-10-13|腾讯科技（深圳）有限公司|Demand method, system and device of media file|

---

US7860996B2|2008-05-30|2010-12-28|Microsoft Corporation|Media streaming with seamless ad insertion|

---

US20100011274A1|2008-06-12|2010-01-14|Qualcomm Incorporated|Hypothetical fec decoder and signalling for decoding control|

---

US8775566B2|2008-06-21|2014-07-08|Microsoft Corporation|File format for media distribution and presentation|

---

US8387150B2|2008-06-27|2013-02-26|Microsoft Corporation|Segmented media content rights management|

---

US8468426B2|2008-07-02|2013-06-18|Apple Inc.|Multimedia-aware quality-of-service and error correction provisioning|

---

US8539092B2|2008-07-09|2013-09-17|Apple Inc.|Video streaming using multiple channels|

---

US20100153578A1|2008-07-16|2010-06-17|Nokia Corporation|Method and Apparatus for Peer to Peer Streaming|

---

JP2010046838A|2008-08-20|2010-03-04|Brother Ind Ltd|Image recording device|

---

US8638796B2|2008-08-22|2014-01-28|Cisco Technology, Inc.|Re-ordering segments of a large number of segmented service flows|

---

KR101019634B1|2008-09-04|2011-03-07|에스케이 텔레콤주식회사|Media streaming system and method|

---

BRPI0918065A2|2008-09-05|2015-12-01|Thomson Licensing|method and system for dynamic playlist modification.|

---

US8325796B2|2008-09-11|2012-12-04|Google Inc.|System and method for video coding using adaptive segmentation|

---

US8265140B2|2008-09-30|2012-09-11|Microsoft Corporation|Fine-grained client-side control of scalable media delivery|

---

US8370520B2|2008-11-24|2013-02-05|Juniper Networks, Inc.|Adaptive network content delivery system|

---

WO2010078281A2|2008-12-31|2010-07-08|Apple Inc.|Real-time or near real-time streaming|

---

US8099476B2|2008-12-31|2012-01-17|Apple Inc.|Updatable real-time or near real-time streaming|

---

US8743906B2|2009-01-23|2014-06-03|Akamai Technologies, Inc.|Scalable seamless digital video stream splicing|

---

CN102365869B|2009-01-26|2015-04-29|汤姆森特许公司|Frame packing for video coding|

---

US20100189182A1|2009-01-28|2010-07-29|Nokia Corporation|Method and apparatus for video coding and decoding|

---

EP2392144A1|2009-01-29|2011-12-07|Dolby Laboratories Licensing Corporation|Methods and devices for sub-sampling and interleaving multiple images, eg stereoscopic|

---

US20100211690A1|2009-02-13|2010-08-19|Digital Fountain, Inc.|Block partitioning for a data stream|

---

US9281847B2|2009-02-27|2016-03-08|Qualcomm Incorporated|Mobile reception of digital video broadcasting—terrestrial services|

---

US8621044B2|2009-03-16|2013-12-31|Microsoft Corporation|Smooth, stateless client media streaming|

---

US8909806B2|2009-03-16|2014-12-09|Microsoft Corporation|Delivering cacheable streaming media presentations|

---

WO2010120804A1|2009-04-13|2010-10-21|Reald Inc.|Encoding, decoding, and distributing enhanced resolution stereoscopic video|

---

US9807468B2|2009-06-16|2017-10-31|Microsoft Technology Licensing, Llc|Byte range caching|

---

US8903895B2|2009-07-22|2014-12-02|Xinlab, Inc.|Method of streaming media to heterogeneous client devices|

---

US8355433B2|2009-08-18|2013-01-15|Netflix, Inc.|Encoding video streams for adaptive video streaming|

---

CN102835150B|2009-09-02|2015-07-15|苹果公司|MAC packet data unit construction for wireless systems|

---

US9917874B2|2009-09-22|2018-03-13|Qualcomm Incorporated|Enhanced block-request streaming using block partitioning or request controls for improved client-side handling|

---

US20110096828A1|2009-09-22|2011-04-28|Qualcomm Incorporated|Enhanced block-request streaming using scalable encoding|

---

US9438861B2|2009-10-06|2016-09-06|Microsoft Technology Licensing, Llc|Integrating continuous and sparse streaming data|

---

JP2011087103A|2009-10-15|2011-04-28|Sony Corp|Provision of content reproduction system, content reproduction device, program, content reproduction method, and content server|

---

PL2497267T3|2009-11-03|2015-02-27|Ericsson Telefon Ab L M|Streaming with optional broadcast delivery of data segments|

---

WO2011057012A1|2009-11-04|2011-05-12|Huawei Technologies Co., Ltd|System and method for media content streaming|

---

KR101786050B1|2009-11-13|2017-10-16|삼성전자 주식회사|Method and apparatus for transmitting and receiving of data|

---

KR101786051B1|2009-11-13|2017-10-16|삼성전자 주식회사|Method and apparatus for data providing and receiving|

---

CN101729857A|2009-11-24|2010-06-09|中兴通讯股份有限公司|Method for accessing video service and video playing system|

---

WO2011070552A1|2009-12-11|2011-06-16|Nokia Corporation|Apparatus and methods for describing and timing representations in streaming media files|

---

AU2011218489B2|2010-02-19|2015-08-13|Telefonaktiebolaget L M Ericsson |Method and arrangement for adaption in HTTP streaming|

---

EP2537318A4|2010-02-19|2013-08-14|Ericsson Telefon Ab L M|Method and arrangement for representation switching in http streaming|

---

JP5071495B2|2010-03-04|2012-11-14|ウシオ電機株式会社|Light source device|

---

EP3783822A1|2010-03-11|2021-02-24|Electronics and Telecommunications Research Institute|Method and apparatus for transceiving data in a mimo system|

---

US9225961B2|2010-05-13|2015-12-29|Qualcomm Incorporated|Frame packing for asymmetric stereo video|

---

US9497290B2|2010-06-14|2016-11-15|Blackberry Limited|Media presentation description delta file for HTTP streaming|

---

EP2585947A1|2010-06-23|2013-05-01|Telefónica, S.A.|A method for indexing multimedia information|

---

US8918533B2|2010-07-13|2014-12-23|Qualcomm Incorporated|Video switching for streaming video data|

---

US9185439B2|2010-07-15|2015-11-10|Qualcomm Incorporated|Signaling data for multiplexing video components|

---

US9131033B2|2010-07-20|2015-09-08|Qualcomm Incoporated|Providing sequence data sets for streaming video data|

---

KR20120010089A|2010-07-20|2012-02-02|삼성전자주식회사|Method and apparatus for improving quality of multimedia streaming service based on hypertext transfer protocol|

---

US9596447B2|2010-07-21|2017-03-14|Qualcomm Incorporated|Providing frame packing type information for video coding|

---

US8711933B2|2010-08-09|2014-04-29|Sony Computer Entertainment Inc.|Random access point formation using intra refreshing technique in video coding|

---

US9456015B2|2010-08-10|2016-09-27|Qualcomm Incorporated|Representation groups for network streaming of coded multimedia data|

---

KR101737325B1|2010-08-19|2017-05-22|삼성전자주식회사|Method and apparatus for reducing decreasing of qualitly of experience in a multimedia system|

---

US8615023B2|2010-10-27|2013-12-24|Electronics And Telecommunications Research Institute|Apparatus and method for transmitting/receiving data in communication system|

---

US20120151302A1|2010-12-10|2012-06-14|Qualcomm Incorporated|Broadcast multimedia storage and access using page maps when asymmetric memory is used|

---

US8958375B2|2011-02-11|2015-02-17|Qualcomm Incorporated|Framing for an improved radio link protocol including FEC|

---

US20120208580A1|2011-02-11|2012-08-16|Qualcomm Incorporated|Forward error correction scheduling for an improved radio link protocol|

---

US9270299B2|2011-02-11|2016-02-23|Qualcomm Incorporated|Encoding and decoding using elastic codes with flexible source block mapping|

---

US9253233B2|2011-08-31|2016-02-02|Qualcomm Incorporated|Switch signaling methods providing improved switching between representations for adaptive HTTP streaming|

---

US9843844B2|2011-10-05|2017-12-12|Qualcomm Incorporated|Network streaming of media data|

---

US9294226B2|2012-03-26|2016-03-22|Qualcomm Incorporated|Universal object delivery and template-based file delivery|US5292127C1|1992-10-02|2001-05-22|Arcade Planet Inc|Arcade game|

---

US7832727B1|1992-10-02|2010-11-16|Bally Gaming Inc.|Illuminated wheel indicators|

---

US7766329B1|1992-10-02|2010-08-03|Sierra Design Group|Wheel indicator and ticket dispenser apparatus|

---

US6307487B1|1998-09-23|2001-10-23|Digital Fountain, Inc.|Information additive code generator and decoder for communication systems|

---

US7068729B2|2001-12-21|2006-06-27|Digital Fountain, Inc.|Multi-stage code generator and decoder for communication systems|

---

US9419749B2|2009-08-19|2016-08-16|Qualcomm Incorporated|Methods and apparatus employing FEC codes with permanent inactivation of symbols for encoding and decoding processes|

---

US9288010B2|2009-08-19|2016-03-15|Qualcomm Incorporated|Universal file delivery methods for providing unequal error protection and bundled file delivery services|

---

US9240810B2|2002-06-11|2016-01-19|Digital Fountain, Inc.|Systems and processes for decoding chain reaction codes through inactivation|

---

AU2003277198A1|2002-10-05|2004-05-04|Digital Fountain, Inc.|Systematic encoding and decoding of chain reaction codes|

---

EP2722995A3|2003-10-06|2018-01-17|Digital Fountain, Inc.|Soft-decision decoding of multi-stage chain reaction codes|

---

US7775870B2|2003-11-21|2010-08-17|Sierra Design Group|Arcade game|

---

EP1743431A4|2004-05-07|2007-05-02|Digital Fountain Inc|File download and streaming system|

---

US9380096B2|2006-06-09|2016-06-28|Qualcomm Incorporated|Enhanced block-request streaming system for handling low-latency streaming|

---

US9386064B2|2006-06-09|2016-07-05|Qualcomm Incorporated|Enhanced block-request streaming using URL templates and construction rules|

---

US9178535B2|2006-06-09|2015-11-03|Digital Fountain, Inc.|Dynamic stream interleaving and sub-stream based delivery|

---

US9209934B2|2006-06-09|2015-12-08|Qualcomm Incorporated|Enhanced block-request streaming using cooperative parallel HTTP and forward error correction|

---

KR101292851B1|2006-02-13|2013-08-02|디지털 파운튼, 인크.|Streaming and buffering using variable fec overhead and protection periods|

---

US9270414B2|2006-02-21|2016-02-23|Digital Fountain, Inc.|Multiple-field based code generator and decoder for communications systems|

---

US7971129B2|2006-05-10|2011-06-28|Digital Fountain, Inc.|Code generator and decoder for communications systems operating using hybrid codes to allow for multiple efficient users of the communications systems|

---

US8335873B2|2006-09-14|2012-12-18|Opentv, Inc.|Method and systems for data transmission|

---

US9237101B2|2007-09-12|2016-01-12|Digital Fountain, Inc.|Generating and communicating source identification information to enable reliable communications|

---

US9674562B1|2008-12-18|2017-06-06|Vmware, Inc.|Quality evaluation of multimedia delivery in cloud environments|

---

US9336117B2|2010-11-09|2016-05-10|Vmware, Inc.|Remote display performance measurement triggered by application display upgrade|

---

US9214004B2|2008-12-18|2015-12-15|Vmware, Inc.|Watermarking and scalability techniques for a virtual desktop planning tool|

---

US20100211690A1|2009-02-13|2010-08-19|Digital Fountain, Inc.|Block partitioning for a data stream|

---

US9281847B2|2009-02-27|2016-03-08|Qualcomm Incorporated|Mobile reception of digital video broadcasting—terrestrial services|

---

KR101648455B1|2009-04-07|2016-08-16|엘지전자 주식회사|Broadcast transmitter, broadcast receiver and 3D video data processing method thereof|

---

US9917874B2|2009-09-22|2018-03-13|Qualcomm Incorporated|Enhanced block-request streaming using block partitioning or request controls for improved client-side handling|

---

KR101750049B1|2009-11-13|2017-06-22|삼성전자주식회사|Method and apparatus for adaptive streaming|

---

KR101777347B1|2009-11-13|2017-09-11|삼성전자주식회사|Method and apparatus for adaptive streaming based on segmentation|

---

KR101786051B1|2009-11-13|2017-10-16|삼성전자 주식회사|Method and apparatus for data providing and receiving|

---

KR101750048B1|2009-11-13|2017-07-03|삼성전자주식회사|Method and apparatus for providing trick play service|

---

EP2507995A4|2009-12-04|2014-07-09|Sonic Ip Inc|Elementary bitstream cryptographic material transport systems and methods|

---

KR101737084B1|2009-12-07|2017-05-17|삼성전자주식회사|Method and apparatus for streaming by inserting another content to main content|

---

WO2011087449A1|2010-01-18|2011-07-21|Telefonaktiebolaget L M Ericsson |Methods and arrangements for http media stream distribution|

---

KR101777348B1|2010-02-23|2017-09-11|삼성전자주식회사|Method and apparatus for transmitting and receiving of data|

---

US8667164B2|2010-04-26|2014-03-04|Samsung Electronics Co., Ltd.|Method and apparatus for playing live content|

---

KR101652255B1|2010-04-26|2016-09-09|삼성전자주식회사|Method for playing of live contents in broadcasting system|

---

US9253548B2|2010-05-27|2016-02-02|Adobe Systems Incorporated|Optimizing caches for media streaming|

---

US9497290B2|2010-06-14|2016-11-15|Blackberry Limited|Media presentation description delta file for HTTP streaming|

---

KR101702562B1|2010-06-18|2017-02-03|삼성전자 주식회사|Storage file format for multimedia streaming file, storage method and client apparatus using the same|

---

US9049497B2|2010-06-29|2015-06-02|Qualcomm Incorporated|Signaling random access points for streaming video data|

---

US8918533B2|2010-07-13|2014-12-23|Qualcomm Incorporated|Video switching for streaming video data|

---

US9185439B2|2010-07-15|2015-11-10|Qualcomm Incorporated|Signaling data for multiplexing video components|

---

KR20120034550A|2010-07-20|2012-04-12|한국전자통신연구원|Apparatus and method for providing streaming contents|

---

US9596447B2|2010-07-21|2017-03-14|Qualcomm Incorporated|Providing frame packing type information for video coding|

---

US9456015B2|2010-08-10|2016-09-27|Qualcomm Incorporated|Representation groups for network streaming of coded multimedia data|

---

CN102130936B|2010-08-17|2013-10-09|华为技术有限公司|Method and device for supporting time shifting and look back in dynamic hyper text transport protocolstreaming transmission scheme|

---

US9467493B2|2010-09-06|2016-10-11|Electronics And Telecommunication Research Institute|Apparatus and method for providing streaming content|

---

CN103222276B|2010-09-20|2017-04-19|数码士有限公司|Processing method to be implemented upon the occurrence of an expression switch in http streaming|

---

US9369512B2|2010-10-06|2016-06-14|Electronics And Telecommunications Research Institute|Apparatus and method for providing streaming content|

---

JP5640649B2|2010-10-27|2014-12-17|ソニー株式会社|Data communication method and information processing apparatus|

---

US8468262B2|2010-11-01|2013-06-18|Research In Motion Limited|Method and apparatus for updating http content descriptions|

---

GB2499539B|2011-10-27|2017-05-03|Lg Electronics Inc|Method for transreceiving media content and device for transreceiving using same|

---

CN106851334A|2010-11-02|2017-06-13|Lg电子株式会社|Send the method for media content and the device of transmitting-receiving media content|

---

US8788079B2|2010-11-09|2014-07-22|Vmware, Inc.|Monitoring audio fidelity and audio-video synchronization|

---

WO2012065186A2|2010-11-12|2012-05-18|Realnetworks, Inc.|Traffic management in adaptive streaming protocols|

---

US8914534B2|2011-01-05|2014-12-16|Sonic Ip, Inc.|Systems and methods for adaptive bitrate streaming of media stored in matroska container files using hypertext transfer protocol|

---

US9661104B2|2011-02-07|2017-05-23|Blackberry Limited|Method and apparatus for receiving presentation metadata|

---

US9270299B2|2011-02-11|2016-02-23|Qualcomm Incorporated|Encoding and decoding using elastic codes with flexible source block mapping|

---

US8958375B2|2011-02-11|2015-02-17|Qualcomm Incorporated|Framing for an improved radio link protocol including FEC|

---

US11025962B2|2011-02-28|2021-06-01|Adobe Inc.|System and method for low-latency content streaming|

---

US9860293B2|2011-03-16|2018-01-02|Electronics And Telecommunications Research Institute|Apparatus and method for providing streaming content using representations|

---

KR101803970B1|2011-03-16|2017-12-28|삼성전자주식회사|Method and apparatus for composing content|

---

US8813116B2|2011-04-27|2014-08-19|Morega Systems Inc.|Adaptive video server with virtual file system and methods for use therewith|

---

US8510555B2|2011-04-27|2013-08-13|Morega Systems Inc|Streaming video server with virtual file system and methods for use therewith|

---

EP2706755A4|2011-05-27|2014-07-16|Huawei Tech Co Ltd|Media transmission method, media reception method, client and system thereof|

---

KR20120138604A|2011-06-14|2012-12-26|삼성전자주식회사|Method and apparatus for transmitting/receiving hybrid media content in a multimedia system|

---

US8745122B2|2011-06-14|2014-06-03|At&T Intellectual Property I, L.P.|System and method for providing an adjunct device in a content delivery network|

---

CN102843335B|2011-06-20|2015-09-09|华为技术有限公司|The processing method of streaming medium content and equipment|

---

KR101222432B1|2011-07-06|2013-01-15|주식회사에어플러그|Apparatus and method for enabling to transceive data using a plurality of heterogeneous networks selectively through a fixed host address|

---

JP2013021574A|2011-07-12|2013-01-31|Sharp Corp|Generation device, distribution server, generation method, reproduction device, reproduction method, reproduction system, generation program, reproduction program, recording medium, and data structure|

---

JP2013038766A|2011-07-12|2013-02-21|Sharp Corp|Transmitter, transmitter control method, control program, and recording medium|

---

US9590814B2|2011-08-01|2017-03-07|Qualcomm Incorporated|Method and apparatus for transport of dynamic adaptive streaming over HTTPinitialization segment description fragments as user service description fragments|

---

SG2014008775A|2011-08-16|2014-04-28|Destiny Software Productions Inc|Script-based video rendering|

---

WO2013033242A1|2011-08-29|2013-03-07|Latakoo, Inc.|Compressing, transcoding, sending, and retrieving video and audio files in a server-based system|

---

US9253233B2|2011-08-31|2016-02-02|Qualcomm Incorporated|Switch signaling methods providing improved switching between representations for adaptive HTTP streaming|

---

US8806188B2|2011-08-31|2014-08-12|Sonic Ip, Inc.|Systems and methods for performing adaptive bitrate streaming using automatically generated top level index files|

---

US8676952B2|2011-09-13|2014-03-18|Ericsson Television Inc.|User adaptive HTTP stream manager and method for using same|

---

US9445136B2|2011-09-21|2016-09-13|Qualcomm Incorporated|Signaling characteristics of segments for network streaming of media data|

---

EP2759115A4|2011-09-23|2015-05-20|Ericsson Telefon Ab L M|Caching in a telecommunication network|

---

KR101678540B1|2011-09-30|2016-11-22|후아웨이 테크놀러지 컴퍼니 리미티드|Method and device for transmitting streaming media|

---

US9843844B2|2011-10-05|2017-12-12|Qualcomm Incorporated|Network streaming of media data|

---

AU2011232766B2|2011-10-07|2014-03-20|Accenture Global Services Limited|Synchronising digital media content|

---

KR20130040132A|2011-10-13|2013-04-23|한국전자통신연구원|A method of transporting media data independent of media codec through heterogeneous ip network|

---

US9712891B2|2011-11-01|2017-07-18|Nokia Technologies Oy|Method and apparatus for selecting an access method for delivery of media|

---

US9124674B2|2011-12-01|2015-09-01|Futurewei Technologies, Inc.|Systems and methods for connection pooling for video streaming in content delivery networks|

---

EP2793479A4|2011-12-12|2015-07-01|Lg Electronics Inc|Device and method for receiving media content|

---

US20130159382A1|2011-12-15|2013-06-20|Microsoft Corporation|Generically presenting virtualized data|

---

US9319321B2|2011-12-16|2016-04-19|Netflix, Inc.|Web server constraint support|

---

US10218756B2|2012-01-06|2019-02-26|Comcast Cable Communications, Llc|Streamlined delivery of video content|

---

US9930379B2|2012-01-31|2018-03-27|Comcast Cable Communications, Llc|System and method for data stream fragmentation|

---

US9450997B2|2012-02-27|2016-09-20|Qualcomm Incorporated|Dash client and receiver with request cancellation capabilities|

---

US20140082661A1|2012-03-06|2014-03-20|Google Inc.|Low latency video storyboard delivery with selectable resolution levels|

---

US9294226B2|2012-03-26|2016-03-22|Qualcomm Incorporated|Universal object delivery and template-based file delivery|

---

JP6465654B2|2012-06-04|2019-02-06|国立大学法人 東京大学|Network system and access point device|

---

US9571827B2|2012-06-08|2017-02-14|Apple Inc.|Techniques for adaptive video streaming|

---

CN103516731B|2012-06-15|2017-04-19|华为技术有限公司|Cache server service method, cache server, and system|

---

US10616297B2|2012-07-09|2020-04-07|Futurewei Technologies, Inc.|Content-specific identification and timing behavior in dynamic adaptive streaming over hypertext transfer protocol|

---

EP2875417B1|2012-07-18|2020-01-01|Verimatrix, Inc.|Systems and methods for rapid content switching to provide a linear tv experience using streaming content distribution|

---

US9804668B2|2012-07-18|2017-10-31|Verimatrix, Inc.|Systems and methods for rapid content switching to provide a linear TV experience using streaming content distribution|

---

US9628542B2|2012-08-24|2017-04-18|Akamai Technologies, Inc.|Hybrid HTTP and UDP content delivery|

---

TWI516104B|2012-09-04|2016-01-01|緯創資通股份有限公司|Method of playing internet video and related electronic device|

---

US20140074961A1|2012-09-12|2014-03-13|Futurewei Technologies, Inc.|Efficiently Delivering Time-Shifted Media Content via Content Delivery Networks |

---

CN103702237A|2012-09-28|2014-04-02|北京大学|Rate self-adapting method and device for HTTPstreaming media|

---

US8639781B1|2012-10-19|2014-01-28|Dropbox, Inc.|Systems and methods for downloading files|

---

JP6236459B2|2012-10-19|2017-11-22|インターデイジタル パテント ホールディングス インコーポレイテッド|Multiple hypothesis rate adaptation for HTTP streaming|

---

EP2912813B1|2012-10-23|2019-12-04|Telefonaktiebolaget LM Ericsson |A method and apparatus for distributing a media content service|

---

CN104041108A|2012-10-30|2014-09-10|华为技术有限公司|Data Transmission Method, Switching Method, Data Transmission Apparatus, Switching Apparatus, User Equipment, Wireless Access Node, Data Transmission System And Switching System|

---

US9015470B2|2012-11-08|2015-04-21|Morega Systems, Inc|Adaptive video server with fast initialization and methods for use therewith|

---

WO2014083739A1|2012-11-28|2014-06-05|パナソニック株式会社|Receiving terminal and receiving method|

---

US20140164482A1|2012-12-11|2014-06-12|Morega Systems Inc.|Video server with bookmark processing and methods for use therewith|

---

US20150373423A1|2013-01-15|2015-12-24|Sharp Kabushiki Kaisha|Video supply device, video acquisition device, and program|

---

CN104937583B|2013-01-18|2018-09-28|华为技术有限公司|It is a kind of to carry out adaptive method and apparatus to media content|

---

US9432426B2|2013-02-04|2016-08-30|Qualcomm Incorporated|Determining available media data for network streaming|

---

US9992499B2|2013-02-27|2018-06-05|Apple Inc.|Adaptive streaming techniques|

---

US9734194B1|2013-03-14|2017-08-15|Google Inc.|Encoding time interval information|

---

US8826346B1|2013-03-15|2014-09-02|General Instrument Corporation|Methods of implementing trickplay|

---

US20140297882A1|2013-04-01|2014-10-02|Microsoft Corporation|Dynamic track switching in media streaming|

---

US9603039B2|2013-04-03|2017-03-21|Qualcomm Incorporated|Opportunistic media patching for a communication session|

---

US9900166B2|2013-04-12|2018-02-20|Qualcomm Incorporated|Methods for delivery of flows of objects over broadcast/multicast enabled networks|

---

US10284612B2|2013-04-19|2019-05-07|Futurewei Technologies, Inc.|Media quality information signaling in dynamic adaptive video streaming over hypertext transfer protocol|

---

US9973559B2|2013-05-29|2018-05-15|Avago Technologies General IpPte. Ltd.|Systems and methods for presenting content streams to a client device|

---

CN108400946B|2013-06-11|2019-09-17|杭州硕文软件有限公司|It is a kind of for reducing the method, apparatus, system and medium of Internet traffic|

---

DE102013211571B4|2013-06-19|2016-02-11|Opticom Dipl.-Ing. Michael Keyhl Gmbh|CONCEPT FOR DETERMINING THE QUALITY OF A MEDIA DATA FLOW WITH A VARIANT QUALITY-TO-BIT RATE|

---

EP2819379A1|2013-06-28|2014-12-31|Thomson Licensing|Method for adapting the downloading behavior of a client terminal configured to receive multimedia content, and corresponding terminal|

---

KR102024311B1|2013-07-12|2019-09-23|캐논 가부시끼가이샤|Adaptive data streaming method with push messages control|

---

KR102272876B1|2013-07-17|2021-07-05|소니그룹주식회사|Content provision device, content provision method, program, terminal device, and content provision system|

---

US9955203B2|2013-09-24|2018-04-24|Ericsson Ab|Recording device and method for efficient network personal video recorder manipulation through adaptive bit rate streaming|

---

US9270721B2|2013-10-08|2016-02-23|Qualcomm Incorporated|Switching between adaptation sets during media streaming|

---

WO2015068518A1|2013-11-05|2015-05-14|株式会社リコー|Communication device, communication system, communication method, and communication program|

---

CA2926729A1|2013-11-05|2015-05-14|Ricoh Company, Ltd.|Communication apparatus, communication system, communication method, and communication program|

---

JP2015103105A|2013-11-26|2015-06-04|株式会社リコー|Communication device, communication system, and communication program|

---

WO2015104450A1|2014-01-07|2015-07-16|Nokia Technologies Oy|Media encapsulating and decapsulating|

---

US10397664B2|2014-01-10|2019-08-27|Sony Corporation|Method for operating a mobile device|

---

US20150201253A1|2014-01-10|2015-07-16|Samsung Electronics Co., Ltd.|Methods and apparatus for universal presentation timeline alignment|

---

JP2015136057A|2014-01-17|2015-07-27|ソニー株式会社|Communication device, communication data generation method, and communication data processing method|

---

JP2015136060A|2014-01-17|2015-07-27|ソニー株式会社|Communication device, communication data generation method, and communication data processing method|

---

JP2015136059A|2014-01-17|2015-07-27|ソニー株式会社|Communication device, communication data generation method, and communication data processing method|

---

US9900362B2|2014-02-11|2018-02-20|Kiswe Mobile Inc.|Methods and apparatus for reducing latency shift in switching between distinct content streams|

---

US9342229B2|2014-03-28|2016-05-17|Acast AB|Method for associating media files with additional content|

---

WO2015152569A1|2014-04-01|2015-10-08|유비쿼스|Method for synchronization communication in access network having g.hn technology applied thereto, and access network line concentration instrument, access network terminal and access network system using same|

---

EP3128709B1|2014-04-01|2021-09-08|Ubiquoss Inc.|Method for controlling line in access network having g.hn technology applied thereto, and access network line concentration instrument, access network terminal and access network system using same|

---

US9448789B2|2014-04-04|2016-09-20|Avid Technology, Inc.|Method of consolidating, synchronizing, and streaming production content for distributed editing of media compositions|

---

CN103957471B|2014-05-05|2017-07-14|华为技术有限公司|The method and apparatus that Internet video is played|

---

KR101538114B1|2014-05-23|2015-07-23|가천대학교 산학협력단|Video processing apparatus and method for seamless video playing in a mobile smart device based on multi-codec|

---

US20150350369A1|2014-05-30|2015-12-03|Qualcomm Incorporated|Method For Reducing Pre-Fetching Of Multimedia Streaming Data With Minimal Impact On Playback User Experience|

---

US10306021B1|2014-08-21|2019-05-28|Amazon Technologies, Inc.|Streaming content to multiple clients|

---

US10070156B2|2014-09-22|2018-09-04|Arris Enterprises Llc|Video quality of experience based on video quality estimation|

---

WO2016048200A1|2014-09-23|2016-03-31|Telefonaktiebolaget L M Ericsson |Video tune-in|

---

SG11201702127YA|2014-09-26|2017-04-27|Sony Corp|Information processing device and information processing method|

---

JP2016072858A|2014-09-30|2016-05-09|エヌ・ティ・ティ・コミュニケーションズ株式会社|Media data generation method, media data reproduction method, media data generation device, media data reproduction device, computer readable recording medium and program|

---

CN104469392B|2014-12-19|2018-04-20|北京奇艺世纪科技有限公司|A kind of video file storage method and device|

---

US9860294B2|2014-12-24|2018-01-02|Intel Corporation|Media content streaming|

---

ES2874748T3|2015-01-06|2021-11-05|Divx Llc|Systems and methods for encoding and sharing content between devices|

---

US20160308927A1|2015-04-20|2016-10-20|Qualcomm Incorporated|Further Device Timing Adjustments and Methods for Supporting DASH Over Broadcast|

---

US10929353B2|2015-04-29|2021-02-23|Box, Inc.|File tree streaming in a virtual file system for cloud-based shared content|

---

US9615258B2|2015-05-21|2017-04-04|Nokia Solutions And Networks Oy|Method and apparatus for securing timing packets over untrusted packet transport network|

---

GB2538997A|2015-06-03|2016-12-07|Nokia Technologies Oy|A method, an apparatus, a computer program for video coding|

---

GB2530368B|2015-06-03|2016-08-10|Bridgeworks Ltd|Transmitting data|

---

CN106294193B|2015-06-03|2019-10-15|杭州海康威视系统技术有限公司|Store equipment and the piecemeal storage method based on the storage equipment|

---

GB2539461B|2015-06-16|2020-01-08|Canon Kk|Image data encapsulation|

---

EP3314850A1|2015-06-29|2018-05-02|VID SCALE, Inc.|Dash caching proxy application|

---

US10021187B2|2015-06-29|2018-07-10|Microsoft Technology Licensing, Llc|Presenting content using decoupled presentation resources|

---

JP6258897B2|2015-07-01|2018-01-10|シャープ株式会社|Content acquisition device, content acquisition method, metadata distribution device, and metadata distribution method|

---

CN105117186B|2015-08-13|2018-06-08|小米科技有限责任公司|Multimedia messages methods of exhibiting and device|

---

US10693936B2|2015-08-25|2020-06-23|Qualcomm Incorporated|Transporting coded audio data|

---

JP6551107B2|2015-09-24|2019-07-31|ブラザー工業株式会社|Server apparatus, server program, terminal program, moving image transmission method, moving image display method, communication system|

---

JP6099715B2|2015-09-30|2017-03-22|エヌ・ティ・ティ・コミュニケーションズ株式会社|Streaming media playback apparatus, streaming media playback method, and program|

---

JP2016040919A|2015-10-09|2016-03-24|ソニー株式会社|Information processor, information processing method, and program|

---

US9930427B2|2015-12-21|2018-03-27|Comcast Cable Communications Management, Llc|Providing advanced playback and control functionality to video client|

---

US10129358B2|2016-01-15|2018-11-13|Verizon Digital Media Services Inc.|Partitioned serialized caching and delivery of large files|

---

MX2018009876A|2016-02-16|2018-11-09|Nokia Technologies Oy|Media encapsulating and decapsulating.|

---

US10567461B2|2016-08-04|2020-02-18|Twitter, Inc.|Low-latency HTTP live streaming|

---

KR20180031325A|2016-09-20|2018-03-28|삼성전자주식회사|Method and apparatus for providing data to streaming application in adaptive streaming service|

---

US10187178B2|2016-10-11|2019-01-22|Microsoft Technology Licensing, Llc|Dynamically partitioning media streams|

---

RU2656689C2|2016-11-08|2018-06-06|Федеральное государственное казенное военное образовательное учреждение высшего образования "Краснодарское высшее военное училище имени генерала армии С.М. Штеменко" Министерство обороны Российской Федерации|Method of multi-stream encryption of information and device for its implementation|

---

US10785116B1|2017-01-12|2020-09-22|Electronic Arts Inc.|Computer architecture for asset management and delivery|

---

CN108668179B|2017-03-27|2021-05-14|华为技术有限公司|Transmission method of media index file and related equipment|

---

EP3410728A1|2017-05-30|2018-12-05|Vestel Elektronik Sanayi ve Ticaret A.S.|Methods and apparatus for streaming data|

---

US10284888B2|2017-06-03|2019-05-07|Apple Inc.|Multiple live HLS streams|

---

CN110710322A|2017-06-15|2020-01-17|索尼公司|Transmission device, reception device, transmission method, reception method, and recording medium|

---

US10652166B2|2017-06-27|2020-05-12|Cisco Technology, Inc.|Non-real time adaptive bitrate recording scheduler|

---

US10944804B1|2017-11-22|2021-03-09|Amazon Technologies, Inc.|Fragmentation of time-associated data streams|

---

US10878028B1|2017-11-22|2020-12-29|Amazon Technologies, Inc.|Replicating and indexing fragments of time-associated data streams|

---

US11025691B1|2017-11-22|2021-06-01|Amazon Technologies, Inc.|Consuming fragments of time-associated data streams|

---

US10972807B2|2018-04-06|2021-04-06|Deluxe One Llc|Dynamic watermarking of digital media content at point of transmission|

---

US10904642B2|2018-06-21|2021-01-26|Mediatek Singapore Pte. Ltd.|Methods and apparatus for updating media presentation data|

---

CN109861790A|2019-01-16|2019-06-07|顺丰科技有限公司|Data transmission method and device|

---

CN111510770B|2019-01-30|2021-08-24|上海哔哩哔哩科技有限公司|Method and device for switching definition, computer equipment and readable storage medium|

---

US20210400100A1|2020-06-23|2021-12-23|Tencent America LLC|Bandwidth cap signaling using combo-index segment track in media streaming|

---

CN112565337A|2020-11-06|2021-03-26|北京奇艺世纪科技有限公司|Request transmission method, server, client, system and electronic equipment|

---

CN112711215B|2021-02-04|2022-01-25|杭州并坚科技有限公司|Bus terminal controller, bus communication power supply system and communication power supply method thereof|

法律状态:
2019-01-08| B06F| Objections, documents and/or translations needed after an examination request according [chapter 6.6 patent gazette]|

---

2020-01-07| B06U| Preliminary requirement: requests with searches performed by other patent offices: procedure suspended [chapter 6.21 patent gazette]|

---

2020-03-10| B15K| Others concerning applications: alteration of classification|Free format text: AS CLASSIFICACOES ANTERIORES ERAM: H04L 29/06 , H04N 7/24 Ipc: H04L 29/06 (2006.01), H04N 21/231 (2011.01), H04N |

---

2021-05-04| B09A| Decision: intention to grant [chapter 9.1 patent gazette]|

---

2021-07-20| B16A| Patent or certificate of addition of invention granted [chapter 16.1 patent gazette]|Free format text: PRAZO DE VALIDADE: 20 (VINTE) ANOS CONTADOS A PARTIR DE 22/09/2010, OBSERVADAS AS CONDICOES LEGAIS. PATENTE CONCEDIDA CONFORME ADI 5.529/DF, QUE DETERMINA A ALTERACAO DO PRAZO DE CONCESSAO. |

优先权:

---

申请号 | 申请日 | 专利标题

---

US24476709P| true| 2009-09-22|2009-09-22|

---

US25771909P| true| 2009-11-03|2009-11-03|

---

US61/257,719|2009-11-03|

---

US25808809P| true| 2009-11-04|2009-11-04|

---

US61/258,088|2009-11-04|

---

US28577909P| true| 2009-12-11|2009-12-11|

---

US61/285,779|2009-12-11|

---

US29672510P| true| 2010-01-20|2010-01-20|

---

US61/296,725|2010-01-20|

---

US37239910P| true| 2010-08-10|2010-08-10|

---

US61/372,399|2010-08-10|

---

US12/887,476|US9432433B2|2006-06-09|2010-09-21|Enhanced block-request streaming system using signaling or block creation|

---

PCT/US2010/049842|WO2011038013A2|2009-09-22|2010-09-22|Enhanced block-request streaming system using signaling or block creation|

---